Image retrieval using scene graphs
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
This paper develops a novel framework for semantic image retrieval based on the notion of a scene graph. Our scene graphs represent objects (“man”, “boat”), attributes of objects (“boat is white”) and relationships between objects (“man standing on boat”). We use these scene graphs as queries to retrieve semantically related images. To this end, we design a conditional random field model that reasons about possible groundings of scene graphs to test images. The likelihoods of these groundings are used as ranking scores for retrieval. We introduce a novel dataset of 5,000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval. In particular, we evaluate retrieval using full scene graphs and small scene subgraphs, and show that our method outperforms retrieval methods that use only objects or low-level image features. In addition, we show that our full model can be used to improve object localization compared to baseline methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“Datasets have been a driving force of computer vision algorithms, ranging from classi-fication [64] to object recognition and segmentation [18, 12, 55, 15].”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“Their sys-tem is constrained to the six object classes from KITTI [23] while our system uses an open-world vocabulary; our scene graphs also tend to be more complex than their semantic graphs.”
MethodConditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Research works printed but not linked: 5
- jurafsky, 2000, Speech & language processing
- lampert, 2013, Attribute-based classification for zero-shot learning of object categories, PAMI
- yao, 2011, Classifying actions and measuring action similarity by modeling the mutual context of objects and human poses, a) A 1(D2) D3
- 0, OpenSceneGraph
- 10.1145/1964921.1964929 · DOI · Paper page
Cited by
Items 101–200 of 1,001
Fully Convolutional Scene Graph Generation
Liu, H., Yan, N., Mortazavi, M., & Bhanu, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01138 · Cited by 149
Wu, S., Fei, H., Cao, Y., Bing, L., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.823 · Cited by 148
Fine-Grained Scene Graph Generation with Data Transfer
Zhang, A., Yao, Y., Chen, Q., Ji, W., Liu, Z., Sun, M., & Chua, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_24 · Cited by 145
EGTR: Extracting Graph from Transformer for Scene Graph Generation
Im, J., Nam, J., Park, N., Lee, H., & Park, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02287 · Cited by 144
Prototype-Based Embedding Network for Scene Graph Generation
Zheng, C., Lyu, X., Gao, L., Dai, B., & Song, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02182 · Cited by 143
Foundations of spatial perception for robotics: Hierarchical representations and real-time systems
Hughes, N., Chang, Y., Hu, S., Talak, R., Abdulhai, R., Strader, J., & Carlone, L. (2024) · The International Journal of Robotics Research · doi:10.1177/02783649241229725 · Cited by 145
A Multimodal Framework for the Detection of Hateful MemesPreprint
Lippe, P., Holla, N., Chandra, S., Rajamanickam, S., Antoniou, G., Shutova, E., & Yannakoudakis, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12871 · Cited by 142
Detecting Unseen Visual Relations Using Analogies
Peyre, J., Šivic, J., Laptev, I., & Schmid, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00207 · Cited by 141
Scallop: A Language for Neurosymbolic Programming
Li, Z., Huang, J., & Naik, M. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3591280 · Cited by 141
Social Adaptive Module for Weakly-Supervised Group Activity Recognition
Yan, R., Xie, L., Tang, J., Shu, X., & Tian, Q. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_13 · Cited by 140
Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
Mancini, M., Naeem, M. F., Xian, Y., & Akata, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3163667 · Cited by 138
A survey on bias in visual datasets
Fabbrizzi, S., Papadopoulos, S., Ntoutsi, E., & Kompatsiaris, I. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103552 · Cited by 137
SPICE: Semantic Propositional Image Caption EvaluationPreprint
Anderson, P., Fernando, B., Johnson, M., & Gould, S. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08822 · Cited by 136
Conditional Image-Text Embedding Networks
Plummer, B. A., Kordas, P., Kiapour, M. H., Zheng, S., Piramuthu, R., & Lazebnik, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_16 · Cited by 135
Attend and Interact: Higher-Order Object Interactions for Video Understanding
Ma, C.-Y., Kadav, A., Melvin, I., Kira, Z., AlRegib, G., & Graf, H. P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00710 · Cited by 134
Dysen-VDM: Empowering Dynamics-Aware Text-to-Video Diffusion with LLMs
Fei, H., Wu, S., Ji, W., Zhang, H., & Chua, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00730 · Cited by 134
Deep Adversarial Graph Attention Convolution Network for Text-Based Person Search
Liu, J., Zha, Z.-J., Hong, R., Wang, M., & Zhang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350991 · Cited by 126
Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs
Dhamo, H., Manhardt, F., Navab, N., & Tombari, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01604 · Cited by 126
Relationformer: A Unified Framework for Image-to-Graph Generation
Shit, S., Koner, R., Wittmann, B., Paetzold, J. C., Ezhov, I., Li, H., Pan, J., Sharifzadeh, S., Kaissis, G. A., Tresp, V., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_24 · Cited by 124
Learning of Visual Relations: The Devil is in the Tails
Desai, A., Wu, T.-Y., Tripathi, S., & Vasconcelos, N. M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01512 · Cited by 123
Describing Like Humans: On Diversity in Image Captioning
Wang, Q., & Chan, A. B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00432 · Cited by 122
Fei, H., Liu, Q., Zhang, M., Zhang, M., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.329 · Cited by 121
Hyperbolic Contrastive Learning for Visual Representations beyond Objects
Ge, S., Mishra, S. K., Kornblith, S., Li, C., & Jacobs, D. W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00661 · Cited by 119
Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration
Huang, D.-A., Nair, S., Xu, D., Zhu, Y., Garg, A., Fei-Fei, L., Savarese, S., & Niebles, J. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00876 · Cited by 118
Self-citationNeural Graph Matching Networks for Fewshot 3D Action Recognition
Guo, M., Chou, E., Huang, D.-A., Song, S., Yeung, S. Y., & Fei-Fei, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_40 · Cited by 118
Self-citationSemantic–geometric visual place recognition: a new perspective for reconciling opposing views
Garg, S., Sünderhauf, N., & Milford, M. J. (2019) · The International Journal of Robotics Research · doi:10.1177/0278364919839761 · Cited by 116
Yuan, Z., Yan, X., Liao, Y., Zhang, R., Wang, S., Li, Z., & Cui, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00181 · Cited by 115
Learning from Collective Intelligence: Feature Learning Using Social Images and Tags
Zhang, H., Shang, X., Luan, H., Wang, M., & Chua, T.-S. (2017) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/2978656 · Cited by 113
Ravichandran, Z., Peng, L., Hughes, N., Griffith, J., & Carlone, L. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812179 · Cited by 113
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Li, R., Zhang, S., Lin, D., Chen, K., & He, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02652 · Cited by 114
Exploring Context and Visual Pattern of Relationship for Scene Graph Generation
Wang, W., Wang, R., Shan, S., & Chen, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00838 · Cited by 111
Improving Image Captioning with Better Use of CaptionsPreprint
Shi, Z., Xu, Z., Qiu, X., & Zhu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11807 · Cited by 111
Kim, U.-H., Park, J.-M., Song, T.-J., & Kim, J.-H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2931042 · Cited by 110
Context-aware Scene Graph Generation with Seq2Seq Transformers
Lu, Y., Rai, H., Chang, J. S., Knyazev, B. A., Yu, G., Shekhar, S., Taylor, G. W., & Volkovs, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01563 · Cited by 110
Li, Y., Wang, L., Wang, T., Yang, X., Luo, J., Wang, Q., Deng, Y., Wang, W., Sun, X., Li, H., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3508072 · Cited by 110
Krishna, R., Chami, I., Bernstein, M. S., & Fei-Fei, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00718 · Cited by 109
Self-citationRLIP: Relational Language-Image Pre-Training for Human-Object Interaction Detection
Yuan, H., Jiang, J., Albanie, S., Feng, T., Huang, Z., Ni, D., & Tang, M. (2022) · neural information processing systems · doi:10.52202/068431-2712 · Cited by 109
Cross-media analysis and reasoning: advances and directions
Peng, Y., Zhu, W., Zhao, Y., Xu, C., Huang, Q., Lu, H., Zheng, Q., Huang, T., & Gao, W. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1601787 · Cited by 108
VrR-VG: Refocusing Visually-Relevant Relationships
Liang, Y., Bai, Y., Zhang, W., Qian, X., Zhu, L., & Mei, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01050 · Cited by 108
Gordo, A., & Larlus, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.560 · Cited by 107
Chen, S., & Demachi, K. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103619 · Cited by 107
Graph Representation Learning Meets Computer Vision: A Survey
Jiao, L., Chen, J., Liu, F., Yang, S., You, C., Liu, X., Li, L., & Hou, B. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3194869 · Cited by 106
Self-citationSemantics for Robotic Mapping, Perception and Interaction: A Survey
Garg, S., Sünderhauf, N., Dayoub, F., Morrison, D., Cosgun, A., Carneiro, G., Wu, Q., Chin, T.-J., Reid, I., Gould, S., et al. (2020) · Foundations and Trends® in Robotics · doi:10.1561/2300000059 · Cited by 105
Scene graph captioner: Image captioning based on structural visual representation
Xu, N., Liu, A.-A., Liu, J., Nie, W., & Su, Y. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.12.027 · Cited by 104
Wu, S., Fei, H., Ji, W., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.146 · Cited by 104
Panoptic Video Scene Graph Generation
Yang, J., Peng, W., Li, X., Guo, Z., Chen, L., Li, B., Ma, Z., Zhou, K., Zhang, W., Loy, C. C., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01791 · Cited by 104
Structured Sparse R-CNN for Direct Scene Graph Generation
Teng, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01883 · Cited by 103
Contextual Translation Embedding for Visual Relationship Detection and Scene Graph Generation
Hung, Z.-S., Mallya, A., & Lazebnik, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992222 · Cited by 102
Improving Image Captioning with Better Use of Caption
Shi, Z., Xu, Z., Qiu, X., & Zhu, X. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.664 · Cited by 102
Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis
Zhang, C., Yu, J., Song, Y., & Cai, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00958 · Cited by 102
Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-Modal Structured Representations
Huang, Y., Tang, J., Chen, Z., Zhang, R., Zhang, X., Chen, W., Zhao, Z., Zhao, Z., Lv, T., Hu, Z., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28017 · Cited by 103
End-to-End Optimization of Scene Layout
Luo, A., Zhang, Z., Wu, J., & Tenenbaum, J. B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00381 · Cited by 101
Retargetable AR: Context-aware Augmented Reality in Indoor Scenes based on 3D Scene Graph
Tahara, T., Seno, T., Narita, G., & Ishikawa, T. (2020) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct51615.2020.00072 · Cited by 100
PPDL: Predicate Probability Distribution based Loss for Unbiased Scene Graph Generation
Li, W., Zhang, H., Bai, Q., Zhao, G., Jiang, N., & Yuan, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01884 · Cited by 101
CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion
Zhai, G., Örnek, E. P., Wu, S.-C., Di, Y., Tombari, F., Navab, N., & Busam, B. (2023) · neural information processing systems · doi:10.52202/075280-1307 · Cited by 101
Manifesting construction activity scenes via image captioning
Huan, L., Wang, G., Huang, T., He, P., Skitmore, M., & Luo, X. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103334 · Cited by 100
Context Understanding in Computer Vision: A Survey
Wang, X., & Zhu, Z. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103646 · Cited by 100
Spatio-Temporal Action Graph Networks
Herzig, R., Levi, E., Xu, H., Gao, H., Brosh, E., Wang, X., Globerson, A., & Darrell, T. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00288 · Cited by 99
Learning Cross-Modal Context Graph for Visual Grounding
Liu, Y., Wan, B., Zhu, X., & He, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6833 · Cited by 99
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning
Saha, S., Yadav, P., Bauer, L., & Bansal, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.609 · Cited by 99
LLARVA: Vision-Action Instruction Tuning Enhances Robot LearningPreprint
Niu, D., Sharma, Y., Biamby, G., Quenum, J., Bai, Y., Shi, B., Darrell, T., & Herzig, R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.11815 · Cited by 99
Ego-Topo: Environment Affordances From Egocentric Video
Nagarajan, T., Li, Y., Feichtenhofer, C., & Grauman, K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00024 · Cited by 98
Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation
Yang, G., Zhang, J., Zhang, Y., Wu, B., & Yang, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01234 · Cited by 98
Towards Open-Vocabulary Scene Graph Generation with Prompt-Based Finetuning
He, T., Gao, L., Song, J., & Li, Y.-F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19815-1_4 · Cited by 97
RLIPv2: Fast Scaling of Relational Language-Image Pre-training
Yuan, H., Zhang, S., Wang, X., Albanie, S., Pan, Y., Feng, T., Jiang, J., Ni, D., Zhang, Y., & Zhao, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01979 · Cited by 97
The More You Know: Using Knowledge Graphs for Image ClassificationPreprint
Marino, K., Salakhutdinov, R., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.04844 · Cited by 97
Scene Graph Reasoning for Visual Question AnsweringPreprint
Hildebrandt, M. E., Li, H., Koner, R., Tresp, V., & Günnemann, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01072 · Cited by 96
Bayesian Relational Memory for Semantic Visual Navigation
Wu, Y., Wu, Y., Tamar, A., Russell, S., Gkioxari, G., & Tian, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00286 · Cited by 95
Towards goal-oriented semantic signal processing: Applications and future challenges
Kalfa, M., Gok, M., Atalık, A., Tegin, B., Duman, T. M., & Arıkan, O. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2021.103134 · Cited by 94
PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph
Li, Y., Ma, T., Bai, Y., Duan, N., Wei, S., & Wang, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01608 · Cited by 52
Learning Canonical Representations for Scene Graph to Image Generation
Herzig, R., Bar, A., Xu, H., Chechik, G., Darrell, T., & Globerson, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_13 · Cited by 92
Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
Herzig, R., Raboh, M., Chechik, G., Berant, J., & Globerson, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05451 · Cited by 91
Structured Query-Based Image Retrieval Using Scene Graphs
Schroeder, B., & Tripathi, S. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00097 · Cited by 90
Simultaneous Localization And Mapping: Present, Future, and the Robust-Perception Age.Preprint
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I. D., & Leonard, J. J. (2016) · arXiv (Cornell University) · Cited by 90
Leveraging Visual Question Answering for Image-Caption Ranking
Lin, X., & Parikh, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_17 · Cited by 89
EchoScene: Indoor Scene Generation via Information Echo Over Scene Graph Diffusion
Zhai, G., Örnek, E. P., Chen, D. Z., Liao, R., Di, Y., Navab, N., Tombari, F., & Busam, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_10 · Cited by 89
Wang, Y., Xiao, B., Bouferguène, A., Al‐Hussein, M., & Li, H. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101699 · Cited by 87
Knowledge Graphs for Multi-modal Learning: Survey and Perspective
Chen, Z., Zhang, Y., Fang, Y., Geng, Y., Guo, L., Chen, J., Liu, X., Pan, J. Z., Zhang, N., Chen, H., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103124 · Cited by 87
Zhang, Y., Pan, Y., Yao, T., Huang, R., Mei, T., & Chen, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00285 · Cited by 86
Human-Object Interaction from Human-level Instructions
Wu, Z., Li, J., Xu, P., & Liu, C. K. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01040 · Cited by 86
Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-TrainingPreprint
Yang, L., Huang, Z., Song, Y., Hong, S., Li, G., Zhang, W., Cui, B., Ghanem, B., & Yang, M. (2022) · King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) · doi:10.48550/arxiv.2211.11138 · Cited by 86
Panoptic Scene Graph Generation with Semantics-Prototype Learning
Li, L., Ji, W., Wu, Y., Li, M., Qin, Y., Wei, L., & Zimmermann, R. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28098 · Cited by 84
Self-citationFind and Focus: Retrieve and Localize Video Events with Natural Language Queries
Shao, D., Xiong, Y., Zhao, Y., Huang, Q., Qiao, Y., & Lin, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_13 · Cited by 83
4D-OR: Semantic Scene Graphs for OR Domain Modeling
Özsoy, E., Örnek, E. P., Eck, U., Czempiel, T., Tombari, F., & Navab, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16449-1_45 · Cited by 83
Causal Reasoning Meets Visual Representation Learning: A Prospective Study
Liu, Y., Wei, Y.-S., Yan, H., Li, G., & Lin, L. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1362-z · Cited by 83
Visual Object Search by Learning Spatial Context
Druon, R., Yoshiyasu, Y., Kanezaki, A., & Watt, A. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2967677 · Cited by 82
SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks
Monninger, T., Schmidt, J., Rupprecht, J., Raba, D., Jordan, J., Frank, D., Staab, S., & Dietmayer, K. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3234771 · Cited by 82
Storytelling from an Image Stream Using Scene Graphs
Wang, R., Wei, Z., Li, P., Zhang, Q., & Huang, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6455 · Cited by 82
RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic ManipulationPreprint
Jiang, H., Huang, B., Wu, R., Li, Z., Garg, S., Nayyeri, H., Wang, S., & Li, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.15487 · Cited by 82
Aligning Linguistic Words and Visual Semantic Units for Image Captioning
Guo, L., Liu, J., Tang, J., Li, J., Luo, W., & Lu, H. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3350943 · Cited by 81
Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI
Song, Y., Sun, P., Liu, H., Li, Z., Song, W., Xiao, Y., & Zhou, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3399746 · Cited by 81
Explainable goal-driven agents and robots-a comprehensive review
Sado, F., Loo, C. K., Liew, W. S., Kerzel, M., & Wermter, S. (2023) · ACM Computing Surveys · doi:10.1145/3564240 · Cited by 79
Dynamic Scene Graph Generation via Anticipatory Pre-training
Li, Y., Yang, X., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01350 · Cited by 79
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
Yang, X., Zhang, H., & Cai, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_3 · Cited by 77
Interventional Video Relation Detection
Li, Y., Yang, X., Shang, X., & Chua, T. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475540 · Cited by 77
Reasoning With Scene Graphs for Robot Planning Under Partial Observability
Amiri, S., Chandan, K., & Zhang, S. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3157567 · Cited by 77
Auto-encoding and Distilling Scene Graphs for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3042192 · Cited by 77
The Path to Path-Traced Movies
Christensen, P. H., & Jarosz, W. (2016) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000073 · Cited by 77
SAC: Semantic Attention Composition for Text-Conditioned Image Retrieval
Jandial, S., Badjatiya, P., Chawla, P., Chopra, A., Sarkar, M., & Krishnamurthy, B. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00067 · Cited by 76
GraDual: Graph-based Dual-modal Representation for Image-Text Matching
Long, S., Han, S. C., Wan, X., & Poon, J. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00252 · Cited by 76
Metadata
- Authors
- Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Max Planck Institute for Informatics (DE)
- Yahoo (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3668-3678
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2077069816
- Cited by
- 1,101
- References
- 68
- Citations by year
- 2026: 59 · 2025: 84 · 2024: 133 · 2023: 134 · 2022: 130 · 2021: 139 · 2020: 145 · 2019: 117 · 2018: 66 · 2017: 47 · 2016: 33 · 2015: 13
- FWCI
- 21.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- semantic image retrieval, scene graph, conditional random fields, object relations, image ranking, object localization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.