The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Experimental — accuracy not yet measuredIn the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Items 901–1,000 of 1,032
Image-to-Image Retrieval by Learning Similarity between Scene Graphs
Yoon, S., Kang, W. Y., Jeon, S., Lee, S., Han, C., Park, J., & Kim, E. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17281 · Cited by 58
LPF: A Language-Prior Feedback Objective Function for De-biased Visual Question Answering
Liang, Z., Hu, H., & Zhu, J. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462981 · Cited by 49
Hierarchical Feature Aggregation Based on Transformer for Image-Text Matching
Dong, X., Zhang, H., Zhu, L., Nie, L., & Liu, L. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3164230 · Cited by 51
A survey of visual navigation: From geometry to embodied AI
Zhang, T., Hu, X., Xiao, J., & Zhang, G. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105036 · Cited by 53
Knowing What to Learn: A Metric-Oriented Focal Mechanism for Image Captioning
Ji, J., Ma, Y., Sun, X., Zhou, Y., Wu, Y., & Ji, R. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3183434 · Cited by 50
4D-OR: Semantic Scene Graphs for OR Domain Modeling
Özsoy, E., Örnek, E. P., Eck, U., Czempiel, T., Tombari, F., & Navab, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16449-1_45 · Cited by 50
Divide and Conquer: Compositional Experts for Generalized Novel Class Discovery
Yang, M., Zhu, Y., Yu, J., Wu, A., & Deng, C. D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01387 · Cited by 52
Self-citationOvarNet: Towards Open-Vocabulary Object Attribute Recognition
Chen, K., Jiang, X., Hu, Y. G., Tang, X., Gao, Y. F., Chen, J., & Xie, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02252 · Cited by 55
Understanding and Constructing Latent Modality Structures in Multi-Modal Representation Learning
Jiang, Q., Chen, C., Zhao, H., Chen, L., Ping, Q., Tran, S. D., Xu, Y., Zeng, B., & Chilimbi, T. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00740 · Cited by 57
“We follow the previous experiments protocols [13,31] using a union of four datasets for pre-training, which include Conceptual Captions (CC3M) [51], Visual Genome (VG) [28], SBU Captions [45] and COCO [35], constituting 4M unique images and 5M image-text pairs.”
MethodWhat’s “up” with vision-language models? Investigating their struggle with spatial reasoning
Kamath, A., Hessel, J., & Chang, K.-W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.568 · Cited by 187
Compositional Feature Augmentation for Unbiased Scene Graph Generation
Li, L., Chen, G., Xiao, J., Yang, Y., Wang, C., & Chen, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01982 · Cited by 51
“To further demonstrate the performance of each predicate, we displayed the predicate distribution of the training set in the VG dataset [4] and the performance of Motifs [14] and Motifs+CFA on R@100 for each predicate category in Figure 8.”
Self-citationFrom Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Li, R., Zhang, S., Lin, D., Chen, K., & He, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02652 · Cited by 51
“We validate our framework on three SGG benchmarks: Panoptic Scene Graph [40], OpenImages-V6 [16], and Visual Genome [15], achieving state-of-the-art performance in the general open-vocabulary setting.”
BackgroundMethodInfluentialDRAU: Dual Recurrent Attention Units for Visual Question Answering
Osman, A., & Samek, W. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.05.001 · Cited by 55
Joint Commonsense and Relation Reasoning for Image and Video Captioning
Hou, J., Wu, X., Zhang, X., Qi, Y., Jia, Y., & Luo, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6731 · Cited by 50
Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigation
Narasimhan, M., Wijmans, E., Chen, X., Darrell, T., Batra, D., Parikh, D., & Singh, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_30 · Cited by 57
Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering
Tang, R., Ma, C., Zhang, W. E., Wu, Q., & Yang, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58529-7_26 · Cited by 56
Heterogeneous Attention Network for Effective and Efficient Cross-modal Retrieval
Yu, T., Yang, Y., Li, Y., Liu, L., Fei, H., & Li, P. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462924 · Cited by 51
Linguistic Structures as Weak Supervision for Visual Scene Graph Generation
Ye, K., & Kovashka, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00819 · Cited by 50
Survey: Transformer based video-language pre-training
Ruan, L., & Jin, Q. (2022) · AI Open · doi:10.1016/j.aiopen.2022.01.001 · Cited by 56
Learning Physical Graph Representations from Visual ScenesPreprint
Bear, D. M., Fan, C., Mrowca, D., Li, Y., Alter, S., Nayebi, A., Schwartz, J., Fei-Fei, L., Wu, J., Tenenbaum, J. B., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12373 · Cited by 50
Self-citationA Multimodal Framework for the Detection of Hateful MemesPreprint
Lippe, P., Holla, N., Chandra, S., Rajamanickam, S., Antoniou, G., Shutova, E., & Yannakoudakis, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12871 · Cited by 116
From Machine Learning to Robotics: Challenges and Opportunities for Embodied IntelligencePreprint
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D. E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15245 · Cited by 116
HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data
Yu, Q., Li, J., Wei, L., Pang, L., Ye, W., Qin, B., Tang, S., Tian, Q., & Zhuang, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01230 · Cited by 59
“Following previous works [15, 20], we randomly select 500 unique images from the intersection of MSCOCO [19] and Visual Genome [9] for a more detailed evaluation.”
Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network
Li, Y., Pan, Y., Yao, T., Chen, J., & Mei, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17034 · Cited by 49
Visual Question Answering With Dense Inter- and Intra-Modality Interactions
Liu, F., Liu, J., Fang, Z., Hong, R., & Lu, H. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3026892 · Cited by 48
Classification by Attention: Scene Graph Classification with Prior Knowledge
Sharifzadeh, S., Baharlou, S. M., & Volker Tresp, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i6.16636 · Cited by 57
Cui, Y., Zhou, Y., Wang, C., Zhao, Z., Zhang, J., Wang, M., & Yu, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475251 · Cited by 51
Deep Learning Approaches for Fashion Knowledge Extraction From Social Media: A Review
Mameli, M., Paolanti, M., Pietrini, R., Pazzaglia, G., Frontoni, E., & Zingaretti, P. (2021) · IEEE Access · doi:10.1109/access.2021.3137893 · Cited by 53
Intelligent layout generation based on deep generative models: A comprehensive survey
Shi, Y., Shang, M., & Qi, Z. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101940 · Cited by 58
Chen, J., Seng, K. P., Smith, J. S., & Ang, L.-M. (2024) · IEEE Access · doi:10.1109/access.2024.3416370 · Cited by 57
RegionGPT: Towards Region Understanding Vision Language Model
Guo, Q., Mello, S. D., Yin, H., Byeon, W., Cheung, K. C., Yu, Y., Luo, P., & Liu, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01309 · Cited by 74
“We evaluate the region-level captioning ability of our model on the ReferCOCOg [22] and Visual Genome [23], employing the same evaluation metrics as used for image-level captioning: METEOR [3] and CIDEr [45].”
BackgroundMethodInfluentialGroma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
Ma, C., Jiang, Y., Wu, J., Yuan, Z., & Qi, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_24 · Cited by 85
Generating Question Relevant Captions to Aid Visual Question Answering
Wu, J., Hu, Z., & Mooney, R. J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1348 · Cited by 54
Diagnosing the Environment Bias in Vision-and-Language Navigation
Zhang, Y., Tan, H., & Bansal, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/124 · Cited by 51
Cross-Modal Hybrid Feature Fusion for Image-Sentence Matching
Xu, X., Wang, Y., He, Y., Yang, Y., Hanjalić, A., & Shen, H. T. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3458281 · Cited by 50
Learning to generalize to new compositions in image understandingPreprint
Atzmon, Y., Berant, J., Kezami, V., Globerson, A., & Chechik, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07639 · Cited by 49
Zhang, H., Yang, D., Wang, H., Zhao, B., Lan, X., Ding, J., & Zheng, N. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3142401 · Cited by 52
Reasoning With Scene Graphs for Robot Planning Under Partial Observability
Amiri, S., Chandan, K., & Zhang, S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3157567 · Cited by 57
Visual Cluster Grounding for Image Captioning
Jiang, W., Zhu, M., Fang, Y., Shi, G., Zhao, X., & Liu, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3177318 · Cited by 50
Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning
He, T., Gao, L., Song, J., & Li, Y.-F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19815-1_4 · Cited by 56
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model
Ji, Y., Wang, J., Gong, Y., Zhang, L., Zhu, Y., Wang, H., Zhang, J., Sakai, T., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02228 · Cited by 59
Self-citationClover: Towards A Unified Video-Language Alignment and Fusion Model
Huang, J., Li, Y., Feng, J., Wu, X., Sun, X., & Ji, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01427 · Cited by 59
EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment
Shi, C., & Yang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01441 · Cited by 53
“LAION [41] is a large-scale image-text dataset, CC3M represents the Conceptual Caption 3M [5], and CAP-L consists of COCO Caption [29], Visual-Genome [25] and SBU Caption [33].”
MethodVisual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning
Chen, Z., Zhou, Q., Shen, Y., Hong, Y., Sun, Z., Gutfreund, D., & Gan, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27888 · Cited by 48
“We choose the faster R-CNN model (Ren et al. 2015) released by (Han et al. 2021) to detect visual concepts in images, which was trained on Visual Genome (Krishna et al. 2017).”
MethodProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentation
Lan, M., Chen, Z., Ke, Y., Wang, X., Feng, L., & Zhang, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73113-6_5 · Cited by 67
Efficient GPT-4V level multimodal large language model for deployment on edge devices
Yao, Y., Yu, T., Zhang, A., Wang, C., Cui, J., Zhu, H., Cai, T., Chen, C., Li, H., Zhao, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61040-5 · Cited by 60
Learning 3D Scene Synthesis from Annotated RGB‐D Images
Kermani, Z. S., Liao, Z., Tan, P., & Zhang, H. (2016) · Computer Graphics Forum · doi:10.1111/cgf.12976 · Cited by 48
On support relations and semantic scene graphs
Yang, M. Y., Liao, W., Ackermann, H., & Rosenhahn, B. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.07.010 · Cited by 49
Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering
Dong, X., Zhu, L., Zhang, D., Yang, Y., & Wu, F. Q. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240527 · Cited by 47
An End-To-End Network for Generating Social Relationship Graphs
Goel, A., Ma, K. T., & Tan, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01144 · Cited by 48
Hier R-CNN: Instance-Level Human Parts Detection and A New Benchmark
Yang, L., Song, Q., Wang, Z., Hu, M., & Liu, C. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3029901 · Cited by 48
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
Mogadala, A., Kalimuthu, M., & Klakow, D. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11688 · Cited by 146
Weakly-Supervised Video Object Grounding by Exploring Spatio-Temporal Contexts
Yang, X., Liu, X., Jian, M., Gao, X., & Wang, M. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413610 · Cited by 46
DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy Minimization
Tang, Z., Lei, J., & Bansal, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.193 · Cited by 48
Safety compliance checking of construction behaviors using visual question answering
Ding, Y., Liu, M., & Luo, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104580 · Cited by 47
RU-Net: Regularized Unrolling Network for Scene Graph Generation
Lin, X., Ding, C., Zhang, J., Zhan, Y., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01885 · Cited by 47
LocVTP: Video-Text Pre-training for Temporal Localization
Cao, M., Yang, T., Weng, J., Zhang, C., Wang, J., & Zou, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_3 · Cited by 52
Teaching Structured Vision & Language Concepts to Vision & Language Models
Doveh, S., Arbelle, A., Harary, S., Schwartz, E., Herzig, R., Giryes, R., Feris, R., Panda, R., Ullman, S., & Karlinsky, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00261 · Cited by 57
Retrieval-augmented Image Captioning
Ramos, R., Elliott, D., & Martins, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.266 · Cited by 61
Which Tokens to Use? Investigating Token Reduction in Vision Transformers
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00085 · Cited by 47
“Training from scratch on datasets such as ImageNet, OpenImages [26], or Visual Genome [25] would have been prohibitive, and we consider the fine-tuning setting to be more realistic when generalizing to datasets other than ImageNet.”
HyperSOR: Context-Aware Graph Hypernetwork for Salient Object Ranking
Qiao, M., Xu, M., Jiang, L. W., Lei, P., Wen, S., Chen, Y., & Sigal, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3368158 · Cited by 47
“Scene graph annotation: We construct the scene graphs on top of the annotations in VG dataset [32].”
BackgroundMethodIndVisSGG: VLM-based scene graph generation for industrial spatial intelligence
Wang, Z., Yan, Z., Li, S., & Liu, J. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.103107 · Cited by 47
Knowledge Acquisition for Visual Question Answering via Iterative Querying
Zhu, Y., Lim, J. J., & Fei-Fei, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.651 · Cited by 46
Self-citationCLEAR: Cumulative LEARning for One-Shot One-Class Image Recognition
Kozerawski, J., & Turk, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00363 · Cited by 47
Artpedia: A New Visual-Semantic Dataset with Visual and Contextual Sentences in the Artistic Domain
Stefanini, M., Cornia, M., Baraldi, L., Corsini, M., & Cucchiara, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30645-8_66 · Cited by 48
Where and Who? Automatic Semantic-Aware Person Composition
Tan, F., Bernier, C., Cohen, B., Ordóñez, V., & Barnes, C. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00170 · Cited by 47
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation
Wang, W., Wang, R., Shan, S., & Chen, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_14 · Cited by 47
Learning Asynchronous and Sparse Human-Object Interaction in Videos
de Morais, R. F. A. B., Le, V., Venkatesh, S., & Tran, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01578 · Cited by 46
A survey of methods, datasets and evaluation metrics for visual question answering
Sharma, H., & Jalal, A. S. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104327 · Cited by 51
Peng, L., Yang, Y., Wang, Z., Wu, X. Y., & Huang, Z. H. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350925 · Cited by 45
Exemplar-Based Open-Set Panoptic Segmentation Network
Hwang, J., Oh, S. W., Lee, J., & Han, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00123 · Cited by 46
xGQA: Cross-Lingual Visual Question Answering
Pfeiffer, J., Geigle, G., Kamath, A., Steitz, J.-M. O., Roth, S., Vulić, I., & Gurevych, I. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.196 · Cited by 67
Plug-and-Play Regulators for Image-Text Matching
Diao, H., Zhang, Y., Liu, W., Ruan, X., & Lu, H. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3266887 · Cited by 47
“1) Image Representation: Given an image, the Faster R-CNN [55] model pretrained on Visual Genome [56] is first utilized to detect K salient regions with bottom-up attention [5], followed with a linear layer transforming each region feature into a d-dimensional vector.”
MethodRLIPv2: Fast Scaling of Relational Language-Image Pre-training
Yuan, H., Zhang, S., Wang, X., Albanie, S., Pan, Y., Feng, T., Jiang, J., Ni, D., Zhang, Y., & Zhao, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01979 · Cited by 53
“To pre-train RLIPv2, we utilize VG [38], COCO [53] and Objects365 [69].”
BackgroundMethodGraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs
Gao, G., Liu, W., Chen, A., Geiger, A., & Schölkopf, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02012 · Cited by 46
“To avoid these limitations while still achieving object decomposition, we propose GraphDreamer , which takes a scene graph ( e.g. , [12]) as input and generates a compositional 3D scene.”
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
Chen, G., Shen, L., Shao, R., Deng, X., & Nie, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02506 · Cited by 73
“They convert existing datasets equipped with spatial coordinates, like Visual Genome [20] and RefCOCO [19], into the textual instruction format and perform instruction tuning on MLLMs.”
MethodLearning multi-modal representations by watching hundreds of surgical video lectures
Yuan, K., Srivastav, V., Yu, T., Lavanchy, J. L., Marescaux, J., Mascagni, P., Navab, N., & Padoy, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103644 · Cited by 82
HCVRD: A Benchmark for Large-Scale Human-Centered Visual Relationship Detection
Zhuang, B., Wu, Q., Shen, C., Reid, I. D., & van den Hengel, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12260 · Cited by 47
Stacked Latent Attention for Multimodal Reasoning
Fan, H., & Zhou, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00118 · Cited by 46
Towards Automatic Report Generation in Spine Radiology Using Weakly Supervised Framework
Han, Z., Wei, B., Leung, S., Chung, J., & Li, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_22 · Cited by 46
Integrating Knowledge and Reasoning in Image Understanding
Aditya, S., Yang, Y., & Baral, C. R. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/873 · Cited by 46
Hindi Visual Genome: A Dataset for Multi-Modal English to Hindi Machine Translation
Parida, S., Bojar, O., & Dash, S. R. (2019) · Computación y Sistemas · doi:10.13053/cys-23-4-3294 · Cited by 46
STL-CQA: Structure-based Transformers with Localization and Encoding for Chart Question Answering
Singh, H., & Shekhar, S. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.264 · Cited by 46
Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation
Knyazev, B. A., de Vries, H., Cangea, C., Taylor, G. W., Courville, A. C., & Belilovsky, E. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.99 · Cited by 46
IPGN: Interactiveness Proposal Graph Network for Human-Object Interaction Detection
Wang, H., Jiao, L., Liu, F., Li, L., Liu, X., Ji, D., & Gan, W. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3096333 · Cited by 46
Self-citationExplain Me the Painting: Multi-Topic Knowledgeable Art Description Generation
Bai, Z., Nakashima, Y., & García, N. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00537 · Cited by 50
Visual Relation Grounding in Videos
Xiao, J., Shang, X., Yang, X., Tang, S., & Chua, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_27 · Cited by 46
Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries
Dong, Q., Tu, Z., Liao, H., Zhang, Y., Mahadevan, V., & Soatto, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00353 · Cited by 45
Exploiting Scene Graphs for Human-Object Interaction Detection
He, T., Gao, L., Song, J., & Li, Y.-F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01568 · Cited by 47
Multi-Scale Fine-Grained Alignments for Image and Sentence Matching
Li, W., Wang, Y., Su, Y., Li, X., Liu, A.-A., & Zhang, Y. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3128744 · Cited by 46
An analysis of graph convolutional networks and recent datasets for visual question answering
Yusuf, A. A., Feng, C., & Xianling, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10151-2 · Cited by 49
All You May Need for VQA are Image Captions
Changpinyo, S., Kukliansy, D., Szpektor, I., Chen, X., Ding, N., & Soricut, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.142 · Cited by 67
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning
Xu, Z., Shen, Y., & Huang, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.641 · Cited by 102
Panoptic Video Scene Graph Generation
Yang, J., Peng, W., Li, X., Guo, Z., Chen, L., Li, B., Ma, Z., Zhou, K., Zhang, W., Loy, C. C., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01791 · Cited by 45
Unbiased Scene Graph Generation in Videos
Nag, S., Min, K., Tripathi, S., & Roy–Chowdhury, A. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02184 · Cited by 45
“tion of the image SGG benchmark Visual Genome (VG) [31], research on SGG from single images has evolved significantly, with earlier works addressing image SGG utilizing several ways to aggregate spatial context [37,39,42,62,64] and latest ones [12,33–36,55,56,61] addressing fundamental problems such as preventing bias…”
BackgroundMethodHata, K., Krishna, R., Fei-Fei, L., & Bernstein, M. S. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998248 · Cited by 45
Self-citationGenerating Diverse and Descriptive Image Captions Using Visual Paraphrases
Liu, L., Tang, J., Wan, X., & Guo, Z. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00434 · Cited by 43
Akula, A., Gella, S., Al-Onaizan, Y., Zhu, S. C., & Reddy, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.586 · Cited by 54
Reinforced Structured State-Evolution for Vision-Language Navigation
Chen, J., Gao, C., Meng, E., Zhang, Q., & Liu, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01501 · Cited by 45
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“However, existing large-scale visual language datasets (e.g., Visual Genome [331], Conceptual Cap-tions [332], and LAION-400m [333]) generally suffer from limited size, high annotation noise, insufficient semantic diversity, and expensive annotation.”
Self-citation
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
