The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Items 801–900 of 1,032
VISTA: Visual-Textual Knowledge Graph Representation Learning
Lee, J., Chung, C., Lee, H., Jo, S., & Whang, J. J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.488 · Cited by 64
Context-Aware Embeddings for Automatic Art Analysis
García, N., Renoust, B., & Nakashima, Y. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325028 · Cited by 60
Zero-Shot Object Detection: Joint Recognition and Localization of Novel Concepts
Rahman, S., Khan, S. H., & Porikli, F. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01355-6 · Cited by 58
Erdem, E., Kuyu, M., Yagcioglu, S., Frank, A., Parcalabescu, L., Plank, B., Babii, A., Turuta, O., Erdem, A., Calixto, I., et al. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12918 · Cited by 54
MKVSE: Multimodal Knowledge Enhanced Visual-semantic Embedding for Image-text Retrieval
Feng, D., He, X., & Peng, Y. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3580501 · Cited by 62
Unbiased Scene Graph Generation via Two-Stage Causal Modeling
Sun, S., Zhi, S., Liao, Q., Heikkilä, J., & Liu, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3285009 · Cited by 57
“We evaluate our method on VG150 [13], a subset of the VG dataset [51] that includes the most frequent 150 object categories and 50 relationship classes.”
BackgroundMethodRobust Visual Question Answering: Datasets, Methods, and Future Challenges
Ma, J., Wang, P., Kong, D., Wang, Z., Liu, J., Pei, H., & Zhao, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3366154 · Cited by 64
“Therefore, AVQA may be more challenging than previous datasets [6], [12], [76], [77], [91], [92].”
MethodInfluentialExpressing Visual Relationships via Language
Tan, H. X., Dernoncourt, F., Lin, Z., Bui, T., & Bansal, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1182 · Cited by 57
ICDAR 2019 Competition on Scene Text Visual Question Answering
Biten, A. F., Tito, R., Mafla, A., Gómez, L., Rusiñol, M., Mathew, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00251 · Cited by 66
Situational Fusion of Visual Representation for Visual Navigation
Shen, W., Xu, D., Zhu, Y., Fei-Fei, L., Guibas, L. J., & Savarese, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00297 · Cited by 61
Self-citationLanguage-Agnostic Visual-Semantic Embeddings
Wehrmann, J., Lopes, M. A., Souza, D. M., & Barros, R. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00590 · Cited by 54
Utilizing the platform economy effect through EWOM: Does the platform matter?
Xu, X., & Lee, C. (2020) · International Journal of Production Economics · doi:10.1016/j.ijpe.2020.107663 · Cited by 57
Multimodal Encoder-Decoder Attention Networks for Visual Question Answering
Chen, C., Han, D., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.2975093 · Cited by 59
Does my multimodal model learn cross-modal interactions? It’s harder to tell than you might think!
Hessel, J., & Lee, L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.62 · Cited by 57
Show, Tell, and Polish: Ruminant Decoding for Image Captioning
Guo, L., Liu, J., Lu, S., & Lu, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2951226 · Cited by 55
Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
Monfort, M., Jin, S., Liu, A. H., Harwath, D. F., Feris, R., Glass, J. R., & Oliva, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01463 · Cited by 63
Adaptive Semantic-Enhanced Transformer for Image Captioning
Zhang, J., Fang, Z., Sun, H., & Wang, Z. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3185320 · Cited by 59
HL-Net: Heterophily Learning Network for Scene Graph Generation
Lin, X., Ding, C., Zhan, Y., Li, Z., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01887 · Cited by 56
Wu, S., Fei, H., Ji, W., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.146 · Cited by 56
Berns, F., Rossetto, L., Schoeffmann, K., Beecks, C., & Awad, G. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325051 · Cited by 58
Visual Relationships as Functions:Enabling Few-Shot Scene Graph Prediction
Dornadula, A., Narcomey, A., Krishna, R., Bernstein, M. S., & Fei-Fei, L. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00214 · Cited by 56
Self-citationStorytelling from an Image Stream Using Scene Graphs
Wang, R., Wei, Z., Li, P., Zhang, Q., & Huang, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6455 · Cited by 56
Background Learnable Cascade for Zero-Shot Object Detection
Zheng, Y., Huang, R., Han, C., Huang, X., & Cui, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69535-4_7 · Cited by 55
Zero-Shot Visual Question AnsweringPreprint
Teney, D., & van den Hengel, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05546 · Cited by 69
MultiModalQA: Complex Question Answering over Text, Tables and ImagesPreprint
Talmor, A., Yoran, O., Catav, A., Lahav, D., Wang, Y., Asai, A., Ilharco, G., Hajishirzi, H., & Berant, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06039 · Cited by 149
Integrating Scene Semantic Knowledge into Image Captioning
Wei, H., Li, Z., Huang, F., Zhang, C., Ma, H., & Shi, Z. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3439734 · Cited by 55
Towards Visual Question Answering on Pathology Images
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-short.90 · Cited by 66
Zero-shot Natural Language Video Localization
Nam, J., Ahn, D., Kang, D., Ha, S. J., & Choi, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00150 · Cited by 2
Devil's on the Edges: Selective Quad Attention for Scene Graph Generation
Jung, D., Kim, S., Kim, W. H., & Cho, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01790 · Cited by 55
Hyperbolic Contrastive Learning for Visual Representations beyond Objects
Ge, S., Mishra, S. K., Kornblith, S., Li, C., & Jacobs, D. W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00661 · Cited by 67
HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models
Bakr, E. M., Sun, P., Shen, X., Khan, F. F., Li, L. E., & Elhoseiny, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01834 · Cited by 74
“For easy level, we obtain subject, object, and relationship from Visual Genome [33] and obtain triplets by different combinations.”
MethodSelf-citationQuestion Type Guided Attention in Visual Question Answering
Shi, Y., Furlanello, T., Zha, S., & Anandkumar, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_10 · Cited by 55
The Path to Path-Traced Movies
Christensen, P. H., & Jarosz, W. (2016) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000073 · Cited by 64
Velioğlu, R., & Rose, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12975 · Cited by 116
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining
Zhan, X., Wu, Y., Dong, X., Wei, Y., Lu, M., Zhang, Y., Xu, H., & Liang, X. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01157 · Cited by 54
Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation
Ray, A., Sikka, K., Divakaran, A., Lee, S., & Burachas, G. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1596 · Cited by 66
Wu, X., Chen, Q., Li, W., Xiao, Y., & Hu, B. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414046 · Cited by 55
Shin, A., Ishii, M., & Narihira, T. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01547-8 · Cited by 58
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.195 · Cited by 68
Image Synthesis from Layout with Locality-Aware Mask Adaption
Li, Z., Wu, J., Koh, I., Tang, Y., & Sun, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01356 · Cited by 54
HAIR: Hierarchical Visual-Semantic Relational Reasoning for Video Question Answering
Liu, F., Liu, J., Wang, N., & Lu, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00172 · Cited by 56
Coarse-to-Fine Reasoning for Visual Question Answering
Nguyen, B. X., Do, T., Tran, H., Tjiputra, E., Tran, Q. D., & Nguyen, A. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00502 · Cited by 58
Named Entity and Relation Extraction with Multi-Modal Retrieval
Wang, X., Cai, J., Jiang, Y., Xie, P., Tu, K., & Lü, W. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.437 · Cited by 66
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02193 · Cited by 70
Self-citationContext Disentangling and Prototype Inheriting for Robust Visual Grounding
Tang, W., Li, L., Liu, X., Jin, L., Tang, J., & Li, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339628 · Cited by 54
“The referring expressions are automatically generated based on scene graphs from the Visual Genome dataset [64] by a set of expression templates.”
MethodSelf-citationScaling Vision-Language Models with Sparse Mixture of Experts
Shen, S., Yao, Z., Li, C., Darrell, T., Keutzer, K., & He, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.758 · Cited by 113
Wang, J., Zheng, Z., Chen, Z., Ma, A., & Zhong, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28357 · Cited by 69
Napier, T., Ahn, E., Allen‐Ankins, S., Schwarzkopf, L., & Lee, I. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124220 · Cited by 55
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
Zhang, H., Li, H., Li, F., Ren, T., Zou, X., Liu, S., Huang, S., Gao, J., Leizhang, Li, C., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72775-7_2 · Cited by 101
Improving Visual Relationship Detection Using Semantic Modeling of Scene Descriptions
Baier, S., Ma, Y., & Volker Tresp, C. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68288-4_4 · Cited by 53
Deep Structured Learning for Visual Relationship Detection
Zhu, Y., & Jiang, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12271 · Cited by 56
Self-citationADVISE: Symbolism and External Knowledge for Decoding Advertisements
Ye, K., & Kovashka, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_51 · Cited by 56
Pathological Visual Question AnsweringPreprint
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2020) · doi:10.36227/techrxiv.13127537.v1 · Cited by 64
Context-Dependent Diffusion Network for Visual Relationship Detection
Cui, Z., Xu, C., Zheng, W., & Yang, J. J. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240668 · Cited by 53
CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
Kim, J.-H., Kitaev, N., Chen, X., Rohrbach, M., Zhang, B., Tian, Y., Batra, D., & Parikh, D. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1651 · Cited by 75
PFAN++: Bi-Directional Image-Text Retrieval With Position Focused Attention Network
Wang, Y., Yang, H., Bai, X., Qian, X., Ma, L., Lu, J. J., Li, B., & Fan, X. N. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3024822 · Cited by 56
Sparse co-attention visual question answering networks based on thresholds
Guo, Z., & Han, D. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03559-4 · Cited by 53
Grounding Answers for Visual Questions Asked by Visually Impaired People
Chen, C., Anjum, S., & Gurari, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01851 · Cited by 67
From language development to language evolution: A unified view of human lexical creativity
Brochhagen, T., Boleda, G., Gualdoni, E., & Xu, Y. (2023) · Science · doi:10.1126/science.ade7981 · Cited by 53
Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Training
Radenović, F., Dubey, A., Kadian, A., Mihaylov, T., Vandenhende, S., Patel, Y., Wen, Y., Ramanathan, V., & Mahajan, D. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00673 · Cited by 56
Interactive Question Answering Systems: Literature Review
Biancofiore, G. M., Deldjoo, Y., Noia, T. D., Sciascio, E. D., & Narducci, F. (2024) · ACM Computing Surveys · doi:10.1145/3657631 · Cited by 61
Co-Attention Network With Question Type for Visual Question Answering
Yang, C., Jiang, M., Jiang, B., Zhou, W., & Li, K. (2019) · IEEE Access · doi:10.1109/access.2019.2908035 · Cited by 52
Scene Graph Parsing as Dependency Parsing
Wang, Y.-S., Liu, C., Zeng, X., & Yuille, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1037 · Cited by 58
ContextNet: representation and exploration for painting classification and retrieval in context
García, N., Renoust, B., & Nakashima, Y. (2019) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-019-00189-4 · Cited by 56
Answering Questions about Data Visualizations using Efficient Bimodal Fusion
Kafle, K., Shrestha, R., Price, B. L., Cohen, S. D., & Kanan, C. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093494 · Cited by 56
Weakly Supervised Visual Semantic Parsing
Zareian, A., Karaman, S., & Chang, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00379 · Cited by 53
Visual question answering: Which investigated applications?
Barra, S., Bisogni, C., Marsico, M. D., & Ricciardi, S. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.09.008 · Cited by 53
Relation-aware Instance Refinement for Weakly Supervised Visual Grounding
Liu, Y., Wan, B., Ma, L., & He, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00556 · Cited by 58
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence ModelsPreprint
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04870 · Cited by 53
Recall What You See Continually Using GridLSTM in Image Captioning
Wu, L., Xu, M. G., Wang, J., & Perry, S. W. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2931815 · Cited by 51
A survey on generative adversarial network-based text-to-image synthesis
Zhou, R. H., Jiang, C. F., & Xu, Q. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.069 · Cited by 56
Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
Li, L., Lei, J., Gan, Z., & Liu, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00205 · Cited by 64
Self-citationKaramcheti, S., Krishna, R., Fei-Fei, L., & Manning, C. D. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.564 · Cited by 62
Self-citationResistance Training Using Prior Bias: Toward Unbiased Scene Graph Generation
Chen, C., Zhan, Y., Yu, B., Liu, L., Luo, Y. H., & Du, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19896 · Cited by 55
Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels
Pu, T., Chen, T., Wu, H., & Lin, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20105 · Cited by 56
Going Denser with Open-Vocabulary Part Segmentation
Sun, P., Chen, S., Zhu, C., Xiao, F., Luo, P., Xie, S., & Yan, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01417 · Cited by 68
“(2) Even if the alignment could be extracted from the caption, or provided by the dataset annotations [38, 65, 86], we find the caption contains object parts less frequently than objects, as shown in Figure 3.”
BackgroundSelf-citationScene-Driven Multimodal Knowledge Graph Construction for Embodied AI
Song, Y., Sun, P., Liu, H., Li, Z., Song, W., Xiao, Y., & Zhou, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3399746 · Cited by 61
“…utilizes symbolic knowledge from various existing knowledge bases (KBs) such as uni-modal knowledge bases like WordNet [5], ConceptNet [6] and Wiki-data [7] for apperceptive knowledge, or multimodal knowledge bases [8], [9] like ImageNet [10], Visual Genome (VG) [11], etc. for perceptual knowledge.”
BackgroundMethodSceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Jia, B., Chen, Y., Yu, H., Wang, Y., Niu, X., Liu, T., Li, Q., & Huang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_16 · Cited by 89
Chen, Z., Chen, H., Imani, M., Chen, R., & Imani, F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125769 · Cited by 61
Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering
Aditya, S., Yang, Y., & Baral, C. R. (2018) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v32i1.11324 · Cited by 52
GTNet: Generative Transfer Network for Zero-Shot Object Detection
Zhao, S., Gao, C., Shao, Y., Li, L., Yu, C., Ji, Z., & Sang, N. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6996 · Cited by 51
Self-citationBGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation
Dhingra, N., Ritter, F., & Kunz, A. M. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00244 · Cited by 55
Identifying and Compensating for Feature Deviation in Imbalanced Deep LearningPreprint
Ye, H.-J., Chen, H.-Y., Zhan, D., & Chao, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01385 · Cited by 98
Learning Object Interactions and Descriptions for Semantic Image Segmentation
Wang, G., Luo, P., Lin, L., & Wang, X. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.556 · Cited by 51
Temporal Reasoning via Audio Question Answering
Fayek, H. M., & Johnson, J. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3010650 · Cited by 57
Self-citationTowards Knowledge-Aware Video Captioning via Transitive Visual Relationship Detection
Wu, B., Niu, G., Yu, J., Xiao, X., Zhang, J., & Wu, H. R. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3169894 · Cited by 51
HGAN: Hierarchical Graph Alignment Network for Image-Text Retrieval
Guo, J., Wang, M., Zhou, Y., Song, B., Chi, Y., Fan, W., & Chang, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3248160 · Cited by 51
Sharma, H., & Padha, D. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10488-2 · Cited by 51
Composing Object Relations and Attributes for Image-Text Matching
Pham, K., Huynh, C., Lim, S.-N., & Shrivastava, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01361 · Cited by 53
“Attributes describe appearance of objects [22, 36, 38, 39, 44], while relations describe how objects interact with one another [56].”
BackgroundMethodPlant disease recognition datasets in the age of deep learning: challenges and opportunities
Xu, M., Park, J. E., Lee, J., Yang, J., & Yoon, S. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1452551 · Cited by 72
EarthMarker: A Visual Prompting Multimodal Large Language Model for Remote Sensing
Zhang, W., Cai, M., Zhang, T., Zhuang, Y., Li, J., & Mao, X. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3523505 · Cited by 68
“Theoretically, the first key point is establishing unified representation [50]–[52] across different modalities (e.g., text, images, and visual prompts), interpretation levels (e.g., image-level, region-level, and point-level), and a wide range of visual tasks.”
Discriminative Bimodal Networks for Visual Localization and Detection with Natural Language Queries
Zhang, Y., Yuan, L., Guo, Y., He, Z., Huang, I.-A., & Lee, H. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.122 · Cited by 51
Deliberate Attention Networks for Image Captioning
Gao, L., Fan, K., Song, J., Liu, X., Xu, X., & Shen, H. T. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018320 · Cited by 51
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
Kim, D.-J., Choi, J., Oh, T.-H., & Kweon, I. S. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1208 · Cited by 58
Nguyen, V.-Q., Suganuma, M., & Okatani, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_14 · Cited by 54
Deep Relation Embedding for Cross-Modal Retrieval
Zhang, Y., Zhou, W., Wang, M., Tian, Q., & Li, H. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3038354 · Cited by 51
Unpaired Image Captioning With semantic-Constrained Self-Learning
Ben, H., Pan, Y., Li, Y., Yao, T., Hong, R., Wang, M., & Mei, T. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3060948 · Cited by 54
Natural Language Inference from Multiple PremisesPreprint
Lai, A., Bisk, Y., & Hockenmaier, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.02925 · Cited by 52
InterBERT: Vision-and-Language Interaction for Multi-modal PretrainingPreprint
Lin, J., An, Y., Zhang, Y., Liu, J., Zhou, J., & Yang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13198 · Cited by 86
What Artificial Neural Networks Can Tell Us about Human Language Acquisition
Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
