The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Experimental — accuracy not yet measuredIn the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 13×
“32 × 32 pixels) images that were collected using WordNet (Miller 1995) synsets as queries.”
MethodInfluentialShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,252
Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998) · IEEE Intelligent Systems and their Applications · doi:10.1109/5254.708428 · Cited by 7,243
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · cited 1×
“Among those in the second category, a common theme has been to use recurrent neural networks to produce novel captions (Kiros et al. 2014; Mao et al. 2014; Karpathy and Fei-Fei 2015;Vinyals et al. 2015;Chen andLawrenceZitnick 2015; Donahue et al. 2015; Fang et al. 2015).”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Self-citationCIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 3×
“We calculate BLEU, CIDEr, and METEOR scores (Chen et al., 2015) between the generated descriptions and their ground-truth descriptions.”
MethodVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445
LabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,668
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,620
Loper, E., & Bird, S. (2002) · doi:10.3115/1118108.1118117 · Cited by 3,323
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283
Pedestrian Detection: An Evaluation of the State of the Art
Dollár, P., Wojek, C., Schiele, B., & Perona, P. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,232
Scripts, Plans, Goals, and Understanding
Schank, R. C., & Abelson, R. P. (2013) · Psychology Press eBooks · doi:10.4324/9780203781036 · Cited by 3,180
Scripts, Plans, Goals, and Understanding: An Inquiry into Human Knowledge Structures
Hunt, E. B., Schank, R. C., & Abelson, R. P. (1979) · The American Journal of Psychology · doi:10.2307/1421499 · Cited by 2,938
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Baker, C. F., Fillmore, C. J., & Lowe, J. B. (1998) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/980845.980860 · Cited by 2,644
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
A Comparison of Document Clustering Techniques
Steinbach, M. S., Karypis, G., & Kumar, V. (2000) · Cited by 2,532
Forbus, K. D. (1984) · Artificial Intelligence · doi:10.1016/0004-3702(84)90038-9 · Cited by 2,491 · cited 2×
“These efforts have grown tomodel physical processes (Forbus 1984) or tomodel a series of actions as scripts (Schank and Abelson 2013) for stories—both of which are not depicted in a single static image but which play roles in an image’s story (Vedantam et al.”
BackgroundYoung, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00166 · Cited by 2,439
Fei-Fei, L., Fergus, R., & Perona, P. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2005.09.012 · Cited by 2,359
Self-citation(2007) · CaltechAUTHORS (California Institute of Technology) · Cited by 2,274
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Describing objects by their attributes
Farhadi, A., Endres, I., Hoiem, D., & Forsyth, D. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206772 · Cited by 1,917
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
Snow, R., O'Connor, B., Jurafsky, D., & Ng, A. Y. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613751 · Cited by 1,887
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849
Thomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,715
Self-citationMicrosoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · cited 2×
“Keywords Computer Vision · Dataset · Image · Scene Graph · Question Answering · Objects · Attributes · Relationships · Knowledge · Language · Crowdsourcing”
BackgroundLearning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Stefan Harmeling, B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206594 · Cited by 1,567
Building Watson: An Overview of the DeepQA Project
Ferrucci, D. A., Brown, E. W., Chu‐Carroll, J., Fan, J., Gondek, D. C., Kalyanpur, A. A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J. M., et al. (2010) · AI Magazine · doi:10.1609/aimag.v31i3.2303 · Cited by 1,523
Relation Classification via Convolutional Deep Neural Network
Zeng, D., Liu, K., Lai, S., Zhou, G., & Zhao, J. (2015) · Cited by 1,467
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics
Hodosh, M., Young, P., & Hockenmaier, J. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3994 · Cited by 1,375
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,321
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,262
Every Picture Tells a Story: Generating Sentences from Images
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15561-1_2 · Cited by 1,240
Cognition does not affect perception: Evaluating the evidence for “top-down” effects
Firestone, C., & Scholl, B. J. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15000965 · Cited by 1,200
A Statistical Approach to Texture Classification from Single Images
Varma, M., & Zisserman, A. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-4635-4 · Cited by 1,144 · cited 3×
“Next, textures were used to study objects (Varma and Zisserman 2005), while other methods learned to predict colors (Ferrari and Zisserman 2007).”
MethodVisual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M. S., & Fei-Fei, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,118
Self-citationImage retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101
Self-citationA shortest path dependency kernel for relation extraction
Bunescu, R., & Mooney, R. J. (2005) · doi:10.3115/1220575.1220666 · Cited by 971
VQA: Visual Question AnsweringPreprint
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00468 · Cited by 968
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · Cited by 879
VerbNet: A broad-coverage, comprehensive verb lexicon
Schuler, K. K. (2005) · Scholarly Commons (University of Pennsylvania) · doi:10.5281/zenodo.8100881 · Cited by 842
NLTK: The Natural Language Toolkit
Bird, S. (2006) · Proceedings of the COLING/ACL on Interactive presentation sessions - · doi:10.3115/1225403.1225421 · Cited by 841 · cited 3×
“We lowercase, lemmatize (Bird 2006), and strip excess whitespace from all relationships.”
BackgroundInfluentialDependency tree kernels for relation extraction
Culotta, A., & Sorensen, J. S. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1219009 · Cited by 840 · cited 13×
“It is similar to the format used in knowledge bases in NLP.”
BackgroundMethodInfluentialPopowich, F. P. (1999) · Wiley Encyclopedia of Electrical and Electronics Engineering · doi:10.1002/047134608x.w6910 · Cited by 834
Exploring various knowledge in relation extraction
GuoDong, Z., Su, J., Jie, Z., & Min, Z. (2005) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1219840.1219893 · Cited by 767
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 632
Multimodal Neural Language Models
Kiros, R., Salakhutdinov, R., & Zemel, R. (2014) · Cited by 593
Modeling mutual context of object and human pose in human-object interaction activities
Yao, B., & Fei-Fei, L. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540235 · Cited by 574
Self-citationMind's eye: A recurrent visual representation for image caption generation
Chen, X., & Zitnick, C. L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298856 · Cited by 498
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A. K., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 494
Explain Images with Multimodal Recurrent Neural NetworksPreprint
Mao, J., Xu, W., Yang, Y., Wang, J., & Yuille, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.1090 · Cited by 458
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 441
Using corpus statistics and WordNet relations for sense identification
Leacock, C., Miller, G. A., & Chodorow, M. S. (1998) · doi:10.5555/972719.972726 · Cited by 439
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain InputPreprint
Malinowski, M., & Fritz, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0210 · Cited by 432 · cited 2×
“The proposed models range from SVM classifiers (Antol et al., 2015) and probabilistic inference (Malinowski and Fritz, 2014) to recurrent neural networks (Gao et al., 2015, Malinowski et al., 2015, Ren et al., 2015a) and convolutional networks (Ma et al., 2015).”
BackgroundMethodRecognition using visual phrases
Sadeghi, M. A., & Farhadi, A. (2011) · doi:10.1109/cvpr.2011.5995711 · Cited by 421
NEIL: Extracting Visual Knowledge from Web Data
Chen, X., Shrivastava, A., & Gupta, A. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.178 · Cited by 412
Ferrari, V., & Zisserman, A. (2007) · Cited by 398
The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding
Patterson, G., Xu, C., Su, H., & Hays, J. H. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0695-z · Cited by 398
Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval
Schuster, S., Krishna, R., Chang, A., Fei-Fei, L., & Manning, C. D. (2015) · Workshop on Vision and Language (VL) · doi:10.18653/v1/w15-2812 · Cited by 346
Self-citationVisual Turing test for computer vision systems
Geman, D., Geman, S., Hallonquist, N., & Laurent Younes, B. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422953112 · Cited by 319
Word Sense Disambiguation: A Survey
Pal, A. R., & Saha, D. (2015) · International Journal of Control Theory and Computer Modeling · doi:10.5121/ijctcm.2015.5301 · Cited by 314 · cited 3×
“By “canonicalization,” we refer to word sense disambiguation (WSD) by mapping the components in our dataset to their respective synsets in the WordNet ontology (Miller, 1995).”
MethodSalehi, N., Irani, L. C., Bernstein, M. S., Alkhatib, A., Ogbe, E., Milland, K., & Clickhappier (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702508 · Cited by 311
Self-citationAutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes
Rothe, S., & Schütze, H. (2015) · doi:10.3115/v1/p15-1173 · Cited by 279 · cited 1×
“Canonicalization, or WSD (Pal and Saha, 2015), has been used in numerous applications, including machine translation, information retrieval, and information extraction (Rothe and Schütze, 2015, Leacock et al., 1998).”
MethodA Unified Model for Word Sense Representation and Disambiguation
Chen, X., Liu, Z., & Sun, M. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1110 · Cited by 274
Culture and Human Development: A New Look
Bruner, J. S. (2010) · Human Development · doi:10.1159/000276535 · Cited by 268
Learning to Answer Questions from Image Using Convolutional Neural Network
Ma, L., Lu, Z., & Li, H. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10442 · Cited by 260
Zhu, J., Nie, Z., Liu, X., Zhang, B., & Wen, J.-R. (2009) · International World Wide Web Conference (WWW) · doi:10.1145/1526709.1526724 · Cited by 253
Discovering states and transformations in image collections
Isola, P., Lim, J. J., & Adelson, E. H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298744 · Cited by 243 · cited 3×
“objects (Isola et al. 2015) and understand their interactions within a scene (Sadeghi and Farhadi 2011).”
BackgroundWeakly Supervised Learning of Interactions between Humans and Objects
Prest, A., Schmid, C., & Ferrari, V. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.158 · Cited by 225
The SENSEVAL-3 English Lexical Sample Task
Mihalcea, R. F., Chklovski, T. A., & Kilgarriff, A. (2004) · University of North Texas Digital Library (University of North Texas) · Cited by 221
Reasoning about Object Affordances in a Knowledge Base Representation
Zhu, Y., Fathi, A., & Fei-Fei, L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10605-2_27 · Cited by 211
Self-citationBringing Semantics into Focus Using Visual Abstraction
Zitnick, C. L., & Parikh, D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.387 · Cited by 197
Tree Kernel-Based Relation Extraction with Context-Sensitive Structured Parse Tree Information
Zhou, G., Zhang, M., Ji, D., & Zhu, Q. (2007) · Cited by 189
Yao, B., Yang, X., & Zhu, S. C. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74198-5_14 · Cited by 185
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers
Gupta, A. K., & Davis, L. S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88682-2_3 · Cited by 180 · cited 2×
“The scene graph representation has been shown to improve semantic image retrieval (Johnson et al., 2015, Schuster et al., 2015) and image captioning (Farhadi et al., 2009, Chang et al., 2014, Gupta and Davis, 2008).”
MethodUnderstanding Indoor Scenes Using 3D Geometric Phrases
Choi, W., Chao, Y.-W., Pantofaru, C., & Savarese, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.12 · Cited by 176 · cited 2×
“, 2014), and 3D spatial geometry between objects has helped object detection (Choi et al., 2013).”
BackgroundEquilibrated adaptive learning rates for non-convex optimizationPreprint
Dauphin, Y., de Vries, H., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.04390 · Cited by 172 · cited 1×
“We use a learning rate of 0.001 trained with rmsprop (Dauphin et al., 2015).”
MethodInfluentialRecognition by association via learning per-exemplar distances
Malisiewicz, T., & Efros, A. A. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587462 · Cited by 135
Image Question Answering: A Visual Semantic Embedding Model and a New Dataset.Preprint
Ren, M., Kiros, R., & Zemel, R. S. (2015) · arXiv (Cornell University) · Cited by 134
Learning semantic relationships for better action retrieval in images
Ramanathan, V., Li, C., Deng, J., Han, W., Li, Z., Gu, K., Song, Y., Samy Bengio, A., Rosenberg, C., & Fei-Fei, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298713 · Cited by 109
Self-citationNonparametric Part Transfer for Fine-Grained Recognition
Goering, C., Rodner, E., Freytag, A., & Denzler, J. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.319 · Cited by 98
Phrase-based Image CaptioningPreprint
Lebret, R., Pinheiro, P. O., & Collobert, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.03671 · Cited by 98
Zero-Shot Learning via Visual Abstraction
Antol, S., Zitnick, C. L., & Parikh, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_27 · Cited by 95 · cited 1×
“Relationships have already shown their utility in improving visual cognitive tasks (Antol et al. 2014; Yang et al. 2012).”
BackgroundVisKE: Visual knowledge extraction and question answering by visual verification of relation phrases
Sadeghi, F., Divvala, S., & Farhadi, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298752 · Cited by 89
Niu, F., Zhang, C., Ré, C., & SHAVLIK, J. W. (2012) · International Journal on Semantic Web and Information Systems · doi:10.4018/jswis.2012070103 · Cited by 89
Recognizing proxemics in personal photos
Yang, Y., Baker, S., Kannan, A., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248095 · Cited by 87 · cited 1×
“Relationships have already shown their utility in improving visual cognitive tasks (Antol et al. 2014; Yang et al. 2012).”
BackgroundVisual Madlibs: Fill in the blank Image Generation and Question AnsweringPreprint
Yu, L., Park, E., Berg, A. C., & Berg, T. L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00278 · Cited by 75
Embracing Error to Enable Rapid Crowdsourcing
Krishna, R., Hata, K., Chen, S. P., Kravitz, J., Shamma, D. A., Fei-Fei, L., & Bernstein, M. S. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858115 · Cited by 74
Self-citationLearning Common Sense through Visual Abstraction
Vedantam, R., Lin, X., Batra, T., Zitnick, C. L., & Parikh, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.292 · Cited by 72
Incorporating Scene Context and Object Layout into Appearance Modeling
Izadinia, H., Sadeghi, F., & Farhadi, A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.37 · Cited by 49 · cited 2×
“Relative layouts between objects have improved scene categorization (Izadinia et al. 2014), and 3D spatial geometry between objects has helped object detection (Choi et al.”
BackgroundToward Never Ending Language Learning.
Betteridge, J., Carlson, J. A., Hong, S. A., Junior, E. R. H., Law, E., Mitchell, T. M., & Wang, S. H. (2009) · Cited by 41
Semantic Parsing for Text to 3D Scene Generation
Chang, A., Savva, M., & Manning, C. D. (2014) · doi:10.3115/v1/w14-2404 · Cited by 40
Describing Common Human Visual Actions in Images
Ronchi, M. R., & Perona, P. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.52 · Cited by 37
Building a Large-scale Multimodal Knowledge Base System for Answering Visual QueriesPreprint
Zhu, Y., Zhang, C., Ré, C., & Fei-Fei, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.05670 · Cited by 36
Self-citationLearning to Answer Questions From Image Using Convolutional Neural NetworkPreprint
Ma, L., Lu, Z., & Li, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00333 · Cited by 29
A Template-Based Approach Toward Acquisition of Logical Sentences
Hou, C.-S. J., Noy, N. F., & Musen, M. A. (2002) · IFIP advances in information and communication technology · doi:10.1007/978-0-387-35602-0_8 · Cited by 13 · cited 2×
“As a model trained on other datasets would generate complete sentences and would not be comparable (Chen et al., 2015) to our region descriptions, we convert all region descriptions generated by our model into complete sentences using predefined templates (Hou et al., 2002).”
Method
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,416
DOTA: A Large-Scale Dataset for Object Detection in Aerial Images
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00418 · Cited by 3,414
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
