The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Experimental — accuracy not yet measuredIn the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Items 601–700 of 1,032
Room-and-Object Aware Knowledge Reasoning for Remote Embodied Referring Expression
Gao, C., Chen, J., Liu, S., Wang, L., Zhang, Q., & Wu, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00308 · Cited by 88
Representation Learning for Natural Language Processing
Liu, Z., Lin, Y., & Sun, M. (2020) · Springer eBooks · doi:10.1007/978-981-15-5573-2 · Cited by 103
PathVQA: 30000+ Questions for Medical Visual Question Answering
He, X., Zhang, Y., Mou, L., Xing, E., & Xie, P. (2020) · arXiv · doi:10.48550/arxiv.2003.10286 · Cited by 79
Information fusion in visual question answering: A Survey
Zhang, D., Cao, R., & Wu, S. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.03.005 · Cited by 82
Dual Attention Networks for Visual Reference Resolution in Visual Dialog
Kang, G.-C., Lim, J., & Zhang, B. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1209 · Cited by 86
Aligned Dual Channel Graph Convolutional Network for Visual Question Answering
Huang, Q., Wei, J., Cai, Y., Zheng, C., Chen, J., Leung, H.-F., & Li, Q. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.642 · Cited by 79
A Real-Time Global Inference Network for One-Stage Referring Expression Comprehension
Zhou, Y., Ji, R., Luo, G., Sun, X., Su, J., Ding, X., Lin, C., & Tian, Q. C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3090426 · Cited by 85
X -Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning
Yuan, Z., Yan, X., Liao, Y., Guo, Y., Li, G., Cui, S. R., & Li, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00837 · Cited by 87
Wu, S., Fei, H., Cao, Y., Bing, L., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.823 · Cited by 127
Multi-Modal Machine Learning in Engineering Design: A Review and Future Directions
Song, B., Zhou, R., & Ahmed, F. (2023) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4063954 · Cited by 105
Relation DETR: Exploring Explicit Position Relation Prior for Object Detection
Hou, X., Liu, M., Zhang, S., Wei, P., Chen, B., & Lan, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72973-7_6 · Cited by 90
Zero-Shot Object Detection with Textual Descriptions
Li, Z., Yao, L., Zhang, X., Wang, X., Kanhere, S. S., & Zhang, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018690 · Cited by 82
Zero-Shot and Few-Shot Learning With Knowledge Graphs: A Comprehensive Survey
Chen, J., Geng, Y., Chen, Z., Pan, J. Z., He, Y., Zhang, W., Horrocks, I. R., & Chen, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3279374 · Cited by 90
Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation
Yang, G., Zhang, J., Zhang, Y., Wu, B., & Yang, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01234 · Cited by 79
PACO: Parts and Attributes of Common Objects
Ramanathan, V., Kalia, A., Petrovic, V., Wen, Y., Zheng, B., Guo, B., Wang, R., Marquez, A., Kovvuri, R., Kadian, A., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00690 · Cited by 79
Automatic Generation of Grounded Visual Questions
Zhang, S., Qu, L., You, S., Yang, Z., & Zhang, J. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/592 · Cited by 85
Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-To-Image Translation
Tomei, M., Cornia, M., Baraldi, L., & Cucchiara, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00600 · Cited by 79
Application of artificial intelligence algorithms in image processing
Zhang, X., & Dahu, W. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2019.03.004 · Cited by 79
Knowledge Aware Semantic Concept Expansion for Image-Text Matching
Shi, B., Ji, L., Lu, P., Niu, Z., & Duan, N. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/720 · Cited by 81
Visual Scanpath Prediction Using IOR-ROI Recurrent Mixture Density Network
Sun, W., Chen, Z., & Wu, F. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2956930 · Cited by 83
Visual question answering: a state-of-the-art review
Manmadhan, S., & Kovoor, B. C. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09832-7 · Cited by 82
Relation R-CNN: A Graph Based Relation-Aware Network for Object Detection
Chen, S., Li, Z., & Tang, Z. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3025128 · Cited by 78
Self-citationDimensions of commonsense knowledge
Ilievski, F., Oltramari, A., Ma, K., Zhang, B., McGuinness, D. L., & Szekely, P. A. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107347 · Cited by 87
S2 Transformer for Image Captioning
Zeng, P., Zhang, H., Song, J., & Gao, L. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/224 · Cited by 78
Structured Sparse R-CNN for Direct Scene Graph Generation
Teng, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01883 · Cited by 78
X2-VLM: All-in-One Pre-Trained Model for Vision-Language Tasks
Zeng, Y., Zhang, X., Li, H., Wang, J., Zhang, J., & Zhou, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339661 · Cited by 116
Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
Ma, J., Liang, J., Chen, C., & Lu, H. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657469 · Cited by 155
“While for such a laborious labeling task, publicly available datasets, including LVIS [19], ADE20K [67], COCO-stuff [6], Visual Genome [30] and Open Images [32], typically have insufficient image volumes ranging from 10k to 1M or even no text description.”
MethodViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts
Cai, M., Liu, H., Mustikovela, S. K., Meyer, G. P., Chai, Y., Park, D., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01227 · Cited by 88
“ViP-Bench consists of 303 unique image-question pairs, where images are collected from MM-Vet [40], MMBench [18], and Visual Genome [13].”
BackgroundMethodMultimodal Alignment and Fusion: A Survey
Li, S., & Tang, H. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02667-1 · Cited by 267
Identifying Unknown Instances for Autonomous DrivingPreprint
Wong, K., Wang, S., Ren, M., Liang, M., & Urtasun, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11296 · Cited by 81
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 95
Deep image captioning: A review of methods, trends and future challenges
Xu, L., Tang, Q., Lv, J., Zheng, B., Zeng, X., & Li, W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126287 · Cited by 79
Benchmarks for Automated Commonsense Reasoning: A Survey
Davis, E. (2023) · ACM Computing Surveys · doi:10.1145/3615355 · Cited by 101
“ž [77] The images in Visual Genome consisted of all the creative-commons images in the intersection of the preexisting collections MS-COCO [89] and YFCC100M [134].”
BackgroundMethodInfluentialLu, P., Ji, L., Zhang, W., Duan, N., Zhou, M., & Wang, J. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220036 · Cited by 78
Relationship-Embedded Representation Learning for Grounding Referring Expressions
Yang, S., Li, G., & Yu, Y. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2973983 · Cited by 77
Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision
Tan, H. X., & Bansal, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.162 · Cited by 103
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Bain, M., Nagrani, A., Varol, G., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00175 · Cited by 76
Rodis, N., Sardianos, C., Grammatikis, P. I. R., Sarigiannidis, P. G., Varlamis, I., & Papadopoulos, G. T. (2024) · IEEE Access · doi:10.1109/access.2024.3467062 · Cited by 105
“Visual genome (2017) [83] Image, scene graph, attributes Caption See Table I”
BackgroundQi, Y., Pan, Z., Hong, Y., Yang, M.-H., van den Hengel, A., & Wu, Q. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00168 · Cited by 80
Crowdsourcing in Computer Vision
Kovashka, A., Russakovsky, O., Fei-Fei, L., & Grauman, K. L. (2016) · now publishers, Inc. eBooks · doi:10.1561/9781680832136 · Cited by 76
Self-citationAre You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
Hu, R., Fried, D., Rohrbach, A., Klein, D., Darrell, T., & Saenko, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1655 · Cited by 86
Towards goal-oriented semantic signal processing: Applications and future challenges
Kalfa, M., Gok, M., Atalık, A., Tegin, B., Duman, T. M., & Arıkan, O. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2021.103134 · Cited by 81
Adaptive Path Selection for Dynamic Image Captioning
Xian, T., Li, Z., Tang, Z., & Ma, H. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3155795 · Cited by 76
CAAN: Context-Aware attention network for visual question answering
Chen, C., Han, D., & Chang, C. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108980 · Cited by 75
WebQA: Multihop and Multimodal QA
Chang, Y., Cao, G., Narang, M., Gao, J., Suzuki, H., & Bisk, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01600 · Cited by 110
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval
Lu, H., Fei, N., Huo, Y., Gao, Y., Lu, Z., & Wen, J.-R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01524 · Cited by 78
Revealing Single Frame Bias for Video-and-Language Learning
Lei, J., Berg, T. L., & Bansal, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.29 · Cited by 146
Vman: visual-modified attention network for multimodal paradigms
Song, X., Han, D., Chen, C., Shen, X., & Wu, H. (2024) · The Visual Computer · doi:10.1007/s00371-024-03563-4 · Cited by 75
“Visual features are extracted using a pre-trained Faster R-CNN model on Visual Genome [52].”
Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities
Abdali, S., Shaham, S., & Krishnamachari, B. (2024) · ACM Computing Surveys · doi:10.1145/3697349 · Cited by 107
VrR-VG: Refocusing Visually-Relevant Relationships
Liang, Y., Bai, Y., Zhang, W., Qian, X., Zhu, L., & Mei, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01050 · Cited by 83
Multi-Level Multimodal Common Semantic Space for Image-Phrase Grounding
Akbari, H., Karaman, S., Bhargava, S., Chen, B., Vondrick, C., & Chang, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01276 · Cited by 73
Li, W., Yang, Y., Zhang, K., Long, E., He, L., Zhang, L., Zhu, Y., Chen, C., Liu, Z., Wu, X., et al. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-0577-y · Cited by 75
Self-citationAn Improved Attention for Visual Question Answering
Rahman, T., Chou, S.-H., Sigal, L., & Carenini, G. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00181 · Cited by 74
Rajasenbagam, T., Jeyanthi, S., & Pandian, J. A. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03075-2 · Cited by 74
Bilinear Graph Networks for Visual Question Answering
Guo, D., Xu, C., & Tao, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3104937 · Cited by 81
Geometry Attention Transformer with position-aware LSTMs for image captioning
Wang, C., Shen, Y., & Ji, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117174 · Cited by 74
Visuals to Text: A Comprehensive Review on Automatic Image Captioning
Ming, Y., Hu, N., Fan, C., Feng, F., Zhou, J., & Yu, H. L. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105734 · Cited by 85
USER: Unified Semantic Enhancement With Momentum Contrast for Image-Text Retrieval
Zhang, Y., Ji, Z., Wang, D., Pang, Y., & Li, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3348297 · Cited by 78
EGTR: Extracting Graph from Transformer for Scene Graph Generation
Im, J., Nam, J., Park, N., Lee, H., & Park, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02287 · Cited by 74
“To verify the effectiveness of the proposed method, we conduct experiments on two representative SGG datasets: Visual Genome [12] and Open Images V6 [13].”
BackgroundMethodInfluentialBLINK: Multimodal Large Language Models Can See but Not Perceive
Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N. A., Ma, W.-C., & Krishna, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_9 · Cited by 345
Self-citationLearning to Generate Scene Graph from Natural Language Supervision
Zhong, Y., Shi, J., Yang, J., Xu, C., & Li, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00184 · Cited by 73
High-Quality Image Captioning With Fine-Grained and Semantic-Guided Visual Attention
Zhang, Z., Wu, Q., Wang, Y., & Chen, F. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2888822 · Cited by 74
Cross-Modal Image-Text Retrieval with Semantic Consistency
Chen, H., Ding, G., Lin, Z., Zhao, S., & Han, J. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351055 · Cited by 76
KnowIT VQA: Answering Knowledge-Based Questions about Videos
García, N., Otani, M., Chu, C., & Nakashima, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6713 · Cited by 6
Radial Graph Convolutional Network for Visual Question Generation
Xu, X., Wang, T., Yang, Y., Hanjalić, A., & Shen, H. T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2986029 · Cited by 72
Enhancing the alignment between target words and corresponding frames for video captioning
Tu, Y., Zhou, C. S., Guo, J., Gao, S., & Yu, Z. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107702 · Cited by 74
Polysemy Deciphering Network for Human-Object Interaction Detection
Zhong, X., Ding, C., Qu, X., & Tao, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_5 · Cited by 75
Retargetable AR: Context-aware Augmented Reality in Indoor Scenes based on 3D Scene Graph
Tahara, T., Seno, T., Narita, G., & Ishikawa, T. (2020) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct51615.2020.00072 · Cited by 73
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering
Ma, K., Ilievski, F., Francis, J., Bisk, Y., Nyberg, E., & Oltramari, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17593 · Cited by 79
PPDL: Predicate Probability Distribution based Loss for Unbiased Scene Graph Generation
Li, W., Zhang, H., Bai, Q., Zhao, G., Jiang, N., & Yuan, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01884 · Cited by 73
Unified Open-Vocabulary Dense Visual Prediction
Shi, H., Hayat, M., & Cai, J. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3381835 · Cited by 76
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
Liu, J., Chen, S., He, X., Guo, L., Zhu, X., Wang, W., & Tang, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3479776 · Cited by 175
“CC14M is a combination of a series of image-language datasets, including MSCOCO [86], Visual Genome [87], SBU [88], CC3M [17], and CC12M [18], leading to a total of 14 million images or 20 million image-text pairs.”
Self-citationContext and Attribute Grounded Dense Captioning
Yin, G., Sheng, L., Liu, B., Yu, N., Wang, X., & Shao, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00640 · Cited by 75
Zhang, S., Jiang, T., Wang, T., Kuang, K., Zhao, Z., Zhu, J., Yu, J., Yang, H., & Wu, F. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413518 · Cited by 77
Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards
Yang, X., Zhang, H., Jin, D., Liu, Y., Wu, C.-H., Tan, J., Xie, D., Wang, J., & Wang, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_1 · Cited by 74
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Narayanan, A., & Russakovsky, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_43 · Cited by 73
Human-like Controllable Image Captioning with Verb-specific Semantic Roles
Chen, L., Jiang, Z., Xiao, J., & Liu, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01657 · Cited by 73
Towards Accurate Text-based Image Captioning with Content Diversity Exploration
Xu, G., Niu, S., Tan, M., Luo, Y., Du, Q., & Wu, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01245 · Cited by 74
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation
Wang, L., Huang, J. L., Li, Y., Xu, K., Yang, Z., & Yu, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01387 · Cited by 75
Context Understanding in Computer Vision: A Survey
Wang, X., & Zhu, Z. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103646 · Cited by 85
Multi3DRefer: Grounding Text Description to Multiple 3D Objects
Zhang, Y., Gong, Z., & Chang, A. X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01397 · Cited by 72
“A notable exception is the VGPhraseCut [54] dataset, based on Visual Genome [33] using templated phrases where each phrase is grounded to potentially multiple instance segments.”
MethodFoundations of spatial perception for robotics: Hierarchical representations and real-time systems
Hughes, N., Chang, Y., Hu, S., Talak, R., Abdulhai, R., Strader, J., & Carlone, L. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241229725 · Cited by 95
“The seminal works (Johnson et al., 2015; Krishna et al., 2016) advocated the use of 2D scene graphs to perform cognitive tasks such as image search, image captioning, and answering questions.”
InfluentialEfficient multimodal large language models: a survey
Jin, Y., Li, J., Gu, T., Liu, Y., Zhao, B., Lai, J., Gan, Z., Wang, Y., Wang, C., Tan, X., et al. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00099-6 · Cited by 184
“0.005 m TinyGPT-V [15], SVIT [197] MS-COCO [185], Visual Genome [189] Auto.”
InfluentialTarget Adaptive Context Aggregation for Video Scene Graph Generation
Teng, Y., Wang, L., Li, Z., & Wu, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01343 · Cited by 74
Neighbor-view Enhanced Model for Vision and Language Navigation
An, D., Qi, Y., Huang, Y., Wu, Q., Wang, L., & Tan, T. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475282 · Cited by 74
Hierarchical Graph Attention Network for Visual Relationship Detection
Li, M., & Chen, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01390 · Cited by 71
Structured Multimodal Attentions for TextVQA
Gao, C., Zhu, Q., Wang, P., Li, H., Liu, Y., van den Hengel, A., & Wu, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132034 · Cited by 76
Structured Semantic Transfer for Multi-Label Recognition with Partial Labels
Chen, T., Pu, T., Wu, H., Xie, Y., & Lin, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19910 · Cited by 75
Honeybee: Locality-Enhanced Projector for Multimodal LLM
Cha, J., Kang, W., Mun, J., & Roh, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01311 · Cited by 141
“…such as open-ended VQA [16, 20, 31, 42], multiple-choice VQA [39, 48], captioning [5, 26], and referring expression comprehension (visual grounding and grounded captioning) [21, 23, 41, 57]; 2) using multiple datasets for each task; 3) applying a fine-grained but single template for each dataset.”
MethodPathological Visual Question AnsweringPreprint
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2020) · doi:10.36227/techrxiv.13127537 · Cited by 90
Compressing Visual-linguistic Model via Knowledge Distillation
Fang, Z., Wang, J., Hu, X., Wang, L., Yang, Y., & Liu, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00146 · Cited by 81
Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis
Sun, W., & Wu, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3078577 · Cited by 71
BridgeTower: Building Bridges between Encoders in Vision-Language Representation Learning
Xiao, X., Wu, C., Rosenman, S., Lal, V., Che, W., & Duan, N. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26263 · Cited by 91
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
Li, L., Gan, Z., Lin, K., Lin, C.-C., Liu, Z., Liu, C., & Wang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02214 · Cited by 90
Self-citationLi, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 91
Liu, F., Ren, X., Liu, Y., Wang, H., & Sun, X. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1013 · Cited by 73
Focal Visual-Text Attention for Memex Question Answering
Liang, J., Jiang, L., Cao, L., Kalantidis, Y., Li, L.-J., & Hauptmann, A. G. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2890628 · Cited by 70
Self-citationLearning Visual Knowledge Memory Networks for Visual Question Answering
Zhou, S., Zhu, C., Dong, Y., Cai, D., Chen, Y., & Li, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00807 · Cited by 70
Symbiotic Attention with Privileged Information for Egocentric Action Recognition
Wang, X., Wu, Y., Zhu, L., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6907 · Cited by 69
Zhang, W., Yu, J., Zhao, W., & Ran, C. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.006 · Cited by 70
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
