The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Experimental — accuracy not yet measuredIn the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Items 301–400 of 1,032
Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues
Plummer, B. A., Mallya, A., Cervantes, C. M., Hockenmaier, J., & Lazebnik, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.213 · Cited by 168
Language-Conditioned Graph Networks for Relational Reasoning
Hu, R., Rohrbach, A., Darrell, T., & Saenko, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01039 · Cited by 166
Learning a Text-Video Embedding from Incomplete and Heterogeneous DataPreprint
Miech, A., Laptev, I., & Šivic, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02516 · Cited by 242
Hierarchical Multimodal LSTM for Dense Visual-Semantic Embedding
Niu, Z., Zhou, M., Wang, L., Gao, X., & Hua, G. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.208 · Cited by 168
Attentive Relational Networks for Mapping Images to Scene Graphs
Qi, M., Li, W., Yang, Z., Wang, Y., & Luo, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00408 · Cited by 166
Spatial-Temporal Transformer for Dynamic Scene Graph Generation
Cong, Y., Liao, W., Ackermann, H., Rosenhahn, B., & Yang, M. Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01606 · Cited by 176
Context-Aware Multi-View Summarization Network for Image-Text Matching
Qu, L., Liu, M., Cao, D., Nie, L., & Tian, Q. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413961 · Cited by 164
TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text
Singh, A., Pang, G., Toh, M., Huang, J., Galuba, W., & Hassner, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00869 · Cited by 178
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Wang, S., Saharia, C., Montgomery, C., Pont-Tuset, J., Noy, S., Pellegrini, S., Onoe, Y., Laszlo, S., Fleet, D. J., Soricut, R., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01761 · Cited by 202
Cycle-Consistency for Robust Visual Question Answering
Shah, M., Chen, X., Rohrbach, M., & Parikh, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00681 · Cited by 173
RGB-D object detection and semantic segmentation for autonomous manipulation in clutter
Schwarz, M., Milan, A., Periyasamy, A. S., & Behnke, S. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917713117 · Cited by 162
Energy-Based Learning for Scene Graph Generation
Suhail, M., Mittal, A., Siddiquie, B., Broaddus, C., Eledath, J., Medioni, G. G., & Sigal, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01372 · Cited by 162
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Li, Y., Wang, C., & Jia, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_19 · Cited by 403
ArtEmis: Affective Language for Visual Art
Achlioptas, P., Ovsjanikov, M., Haydarov, K., Elhoseiny, M., & Guibas, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01140 · Cited by 161
Zero-Shot Grounding of Objects From Natural Language Queries
Sadhu, A., Chen, K., & Nevatia, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00479 · Cited by 168
SeqTR: A Simple Yet Universal Network for Visual Grounding
Zhu, C., Zhou, Y., Shen, Y., Luo, G., Pan, X., Lin, M., Chen, C., Cao, L., Sun, X., & Ji, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_35 · Cited by 161
Object Level Visual Reasoning in Videos
Baradel, F., Neverova, N., Wolf, C., Mille, J., & Mori, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_7 · Cited by 163
Mutual Attention Inception Network for Remote Sensing Visual Question Answering
Zheng, X., Wang, B., Du, X., & Lu, X. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3079918 · Cited by 159
SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences
Wu, S.-C., Wald, J., Tateno, K., Navab, N., & Tombari, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00743 · Cited by 158
A Survey of Natural Language Generation
Dong, C., Li, Y., Gong, H., Chen, M., Li, J., Shen, Y., & Yang, M. (2022) · ACM Computing Surveys · doi:10.1145/3554727 · Cited by 203
Fine-Grained Recognition in the Wild: A Multi-task Domain Adaptation Approach
Gebru, T., Hoffman, J., & Fei-Fei, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.151 · Cited by 152
Self-citationTowards Safe Weakly Supervised Learning
Li, Y.-F., Guo, L.-Z., & Zhou, Z.-H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2922396 · Cited by 165
Image-Text Embedding Learning via Visual and Textual Semantic Reasoning
Li, K., Zhang, Y., Li, K., Li, Y., & Fu, Y. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148470 · Cited by 165
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic
Tewel, Y., Shalev, Y., Schwartz, I., & Wolf, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01739 · Cited by 194
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text DataPreprint
Qi, D., Su, L., Song, J. L., Cui, E., Bharti, T., & Sacheti, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07966 · Cited by 255
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Self-citationLook Back and Predict Forward in Image Captioning
Qin, Y. X., Du, J., Zhang, Y., & Lu, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00856 · Cited by 153
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
Lundgard, A., & Satyanarayan, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114770 · Cited by 155
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
Yin, G., Sheng, L., Liu, B., Yu, N., Wang, X., Shao, J., & Loy, C. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_20 · Cited by 158
Learning visual relationship and context-aware attention for image captioning
Wang, J., Wang, W., Wang, L., Wang, Z., Feng, D. D., & Tan, T. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107075 · Cited by 156
All in One: Exploring Unified Video-Language Pre-Training
Wang, J., Ge, Y., Yan, R., Ge, Y., Lin, K. Q., Tsutsui, S., Lin, X., Cai, G., Wu, J., Shan, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00638 · Cited by 199
Multimodal datasets: misogyny, pornography, and malignant stereotypes
Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021) · arXiv · doi:10.48550/arxiv.2110.01963 · Cited by 160
PolyFormer: Referring Image Segmentation as Sequential Polygon Generation
Liu, J., Ding, H., Cai, Z., Zhang, Y., Satzoda, R. K., Mahadevan, V., & Manmatha, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01789 · Cited by 173
“To leverage large-scale image-text dataset with grounded box annotations, we first pre-train PolyFormer on the REC task with the combination of Visual Genome [40], RefCOCO [89], RefCOCO+ [89], RefCOCOg [60, 62], and Flickr30k-entities [64].”
MethodSelf-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155
Detecting and Tracking Small and Dense Moving Objects in Satellite Videos: A Benchmark
Yin, Q., Hu, Q., Liu, H., Zhang, F., Wang, Y., Lin, Z., An, W., & Guo, Y. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3130436 · Cited by 167
Long-CLIP: Unlocking the Long-Text Capability of CLIP
Zhang, B., Zhang, P., Dong, X., Zang, Y., & Wang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72983-6_18 · Cited by 257
GRIT: Faster and Better Image Captioning Transformer Using Dual Visual Features
Nguyen, V.-Q., Suganuma, M., & Okatani, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_10 · Cited by 152
Large-Scale Visual Relationship Understanding
Zhang, J., Kalantidis, Y., Rohrbach, M., Paluri, M., Elgammal, A., & Elhoseiny, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019185 · Cited by 159
Self-citationVisual Entailment: A Novel Task for Fine-Grained Image UnderstandingPreprint
Xie, N., Lai, F., Doran, D., & Kadav, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06706 · Cited by 311
Two Causal Principles for Improving Visual Dialog
Qi, J., Niu, Y., Huang, J., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01087 · Cited by 151
Fine-Grained Visual Textual Alignment for Cross-Modal Retrieval Using Transformer Encoders
Messina, N., Amato, G. M., Esuli, A., Falchi, F., Gennaro, C., & Marchand‐Maillet, S. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3451390 · Cited by 163
Pythia v0.1: the Winning Entry to the VQA Challenge 2018Preprint
Jiang, Y. L., Natarajan, V., Chen, X., Rohrbach, M., Batra, D., & Parikh, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09956 · Cited by 205
Video Visual Relation Detection
Shang, X., Ren, T., Guo, J., Zhang, H., & Chua, T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123380 · Cited by 147
Symbolic Graph Reasoning Meets Convolutions
Liang, X., Hu, Z., Zhang, H., Lin, L., & Xing, E. P. (2018) · neural information processing systems · Cited by 151
Context-Aware Visual Policy Network for Fine-Grained Image Captioning
Zha, Z.-J., Liu, D., Zhang, H., Zhang, Y., & Wu, F. H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2909864 · Cited by 151
Joint embedding VQA model based on dynamic word vector
Ma, Z., Zheng, W., Chen, X., & Yin, L. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.353 · Cited by 149
Hierarchical Interactive Multimodal Transformer for Aspect-Based Multimodal Sentiment Analysis
Yu, J., Chen, K., & Xia, R. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3171091 · Cited by 153
“The Faster R-CNN model is pre-trained on Visual Genome dataset [63], which contains 1,600 object categories and 400 attributes.”
MethodTowards generalist foundation model for radiology by leveraging web-scale 2D&3D medical data
Wu, C., Zhang, X., Zhang, Y., Hui, H., Wang, Y., & Xie, W. (2025) · Nature Communications · doi:10.1038/s41467-025-62385-7 · Cited by 429
“Contrary to the natural scenery domain, which boasts numerous large-scale multi-modal datasets such as MMC4 27 , Visual Genome 28 , and LION-5B 29 , the medical domain is somewhat lacking.”
Annotating Objects and Relations in User-Generated Videos
Shang, X., Di, D., Xiao, J., Cao, Y., Yang, X., & Chua, T. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325056 · Cited by 149
Spatial-Aware Graph Relation Network for Large-Scale Object Detection
Xu, H., Jiang, C., Liang, X., & Li, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00952 · Cited by 146
Gupta, T., Schwing, A. G., & Hoiem, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00977 · Cited by 145
More Grounded Image Captioning by Distilling Image-Text Matching Model
Zhou, Y., Wang, M., Liu, D., Hu, Z., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00483 · Cited by 147
TeachText: CrossModal Generalized Distillation for Text-Video Retrieval
Croitoru, I., Bogolin, S.-V., Leordeanu, M., Jin, H., Zisserman, A., Albanie, S., & Liu, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01138 · Cited by 151
Hierarchical Relational Networks for Group Activity Recognition and Retrieval
Ibrahim, M. S., & Mori, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_44 · Cited by 143
Video Pivoting Unsupervised Multi-Modal Machine Translation
Li, M., Huang, P.-Y., Chang, X., Hu, J., Yang, Y., & Hauptmann, A. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3181116 · Cited by 146
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 197
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Li, K., Wang, Y., Li, Y., Wang, Y., He, Y., Wang, L., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01826 · Cited by 143
“(ii) 17M Corpus includes four other image-text datasets: COCO [48], Visual Genome [39], SBU Captions [57], and CC12M [12].”
BackgroundZhang, Z., Zhao, T., Guo, Y., & Yin, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3449154 · Cited by 208
“We have chosen 11 public large-scale English image–text paired datasets to build the PUB11 subset, including LAION2B-en [59], LAION400M [60], LAIONCOCO, COYO700M [61], CC3M [62], CC12M [63], YFCC15M [64], WIT [65], Redcaps [66], SBU [67], and Visual Genome [68].”
MethodHuang, Q., Dong, X., Zhang, P., Wang, B., He, C., Wang, J., Lin, D., Zhang, W., & Yu, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01274 · Cited by 270
Improving Visual Reasoning Through Semantic Representation
Zheng, W., Liu, X., Ni, X., Yin, L., & Yang, B. (2021) · IEEE Access · doi:10.1109/access.2021.3074937 · Cited by 147
Exploring Visual Relationship for Image CaptioningPreprint
Yao, T., Pan, Y., Li, Y., & Mei, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.07041 · Cited by 141
Hierarchical Attention Network for Image Captioning
Wang, W., Chen, Z., & Hu, H. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018957 · Cited by 140
Image Captioning Through Image Transformer
He, S., Liao, W., Tavakoli, H. R., Yang, M. Y., Rosenhahn, B., & Pugeault, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69538-5_10 · Cited by 147
Injecting Semantic Concepts into End-to-End Image Captioning
Fang, Z., Wang, J., Hu, X., Liang, L., Gan, Z., Wang, L., Yang, Y., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01748 · Cited by 146
ImageNet pre-trained models with batch normalizationPreprint
Simon, M., Rodner, E., & Denzler, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01452 · Cited by 157
Multimodal Prompting with Missing Modalities for Visual Recognition
Lee, Y.-L., Tsai, Y.-H., Chiu, W.-C., & Lee, C.-Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01435 · Cited by 156
“Bold number indicates the best performance. trained on several large vision-language datasets (e.g., MS-COCO [20] and Visual Genome [14]) via objectives such as Image Text Matching and Masked Language Modeling.”
MethodAirbert: In-domain Pretraining for Vision-and-Language Navigation
Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00166 · Cited by 139
Self-citationSituation Recognition with Graph Neural Networks
Li, R., Tapaswi, M., Liao, R., Jia, J., Urtasun, R., & Fidler, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.448 · Cited by 143
Comprehensive Image Captioning via Scene Graph Decomposition
Zhong, Y., Wang, L., Chen, J., Yu, D., & Li, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_13 · Cited by 140
Towards local visual modeling for image captioning
Ma, Y., Ji, J., Sun, X., Zhou, Y., & Ji, R. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109420 · Cited by 139
Transformers in computational visual media: A survey
Xu, Y., Wei, H., Lin, M., Deng, Y., Sheng, K., Zhang, M., Tang, F., Dong, W., Huang, F., & Xu, C. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0247-3 · Cited by 144
Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation
Dong, X., Gan, T., Song, X., Wu, J., Cheng, Y., & Nie, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01882 · Cited by 138
DOTA: A Large-scale Dataset for Object Detection in Aerial ImagesPreprint
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M. P., Pelillo, M., & Zhang, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10398 · Cited by 156
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
Liu, X., Li, H., Shao, J., Chen, D., & Wang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_21 · Cited by 137
PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN
Zhang, H., Kyaw, Z., Yu, J., & Chang, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.454 · Cited by 136
Ding, Y., Yu, J., Liu, B., Hu, Y., Cui, M., & Wu, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00503 · Cited by 138
Rodríguez-Opazo, C., Marrese-Taylor, E., Saleh, F. S., Li, H., & Gould, S. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093328 · Cited by 136
Compositional Learning for Human Object Interaction
Kato, K., Li, Y., & Gupta, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_15 · Cited by 133
Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding
Zhou, Y., Yu, J., Xiang, C., Zhao, Z., Tian, Q. C., & Tao, D. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/155 · Cited by 143
Fine-grained Image-text Matching by Cross-modal Hard Aligning Network
Pan, Z., Wu, F., & Zhang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01847 · Cited by 139
“For each input image V , we follow [21] to extract top-K region-level features, with the Faster R-CNN [31] model pre-trained on Visual Genomes [20] using bottom-up and top-down attention (BUTD) [1].”
MethodDeep Learning From Noisy Image Labels With Quality Embedding
Yao, J., Wang, J., Tsang, I. W.-H., Zhang, Y., Sun, J., Zhang, C., & Zhang, R. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2877939 · Cited by 134
Learning Fragment Self-Attention Embeddings for Image-Text Matching
Wu, Y., Wang, S., Song, G., & Huang, Q. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350940 · Cited by 132
Learning by Abstraction: The Neural State MachinePreprint
Hudson, D. A., & Manning, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03950 · Cited by 159
Interactive Visual Grounding of Referring Expressions for Human-Robot Interaction
Shridhar, M., & Hsu, D. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.028 · Cited by 132
Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases
Weikum, G., Dong, X. L., Razniewski, S., & Suchanek, F. M. (2021) · Foundations and Trends in Databases · doi:10.1561/1900000064 · Cited by 161
Region-Object Relation-Aware Dense Captioning via Transformer
Shao, Z., Han, J., Marnerides, D., & Debattista, K. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3152990 · Cited by 134
MPCCT: Multimodal vision-language learning paradigm with context-based compact Transformer
Chen, C., Han, D., & Chang, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110084 · Cited by 134
Learning Dual Semantic Relations With Graph Attention for Image-Text Matching
Wen, K., Gu, X., & Cheng, Q. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3030656 · Cited by 131
Deconfounded Image Captioning: A Causal Retrospect
Yang, X., Zhang, H., & Cai, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3121705 · Cited by 136
Image Synthesis From Reconfigurable Layout and Style
Sun, W., & Wu, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01063 · Cited by 132
TRAR: Routing the Attention Spans in Transformer for Visual Question Answering
Zhou, Y., Ren, T., Zhu, C., Sun, X., Liu, J., Ding, X., Xu, M., & Ji, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00208 · Cited by 133
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Subramanian, S., Merrill, W., Darrell, T., Gardner, M., Singh, S., & Rohrbach, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.357 · Cited by 172
SGTR: End-to-end Scene Graph Generation with Transformer
Li, R., Zhang, S., & He, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01888 · Cited by 130
Mulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132
Recursive Visual Attention in Visual Dialog
Niu, Y., Zhang, H., Zhang, M., Zhang, J., Lu, Z., & Wen, J.-R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00684 · Cited by 131
PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation
Yan, S., Shen, C., Jin, Z., Huang, J., Jiang, R., Chen, Y., & Hua, X.-S. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413722 · Cited by 129
Panoptic Scene Graph Generation
Yang, J., Ang, Y. Z., Guo, Z., Zhou, K., Zhang, W., & Liu, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_11 · Cited by 129
Oscar: Object-Semantics Aligned Pre-training for Vision-Language TasksPreprint
Li, X., Yin, X., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · arXiv (Cornell University) · doi:10.4230/oasics.ldk.2021.36 · Cited by 129
Human Uncertainty Makes Classification More Robust
Peterson, J., Battleday, R., Griffiths, T., & Russakovsky, O. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00971 · Cited by 196
ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors
Kuo, W., Angelova, A., Malik, J., & Lin, T.-Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00930 · Cited by 126
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
