Image retrieval using scene graphs
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
This paper develops a novel framework for semantic image retrieval based on the notion of a scene graph. Our scene graphs represent objects (“man”, “boat”), attributes of objects (“boat is white”) and relationships between objects (“man standing on boat”). We use these scene graphs as queries to retrieve semantically related images. To this end, we design a conditional random field model that reasons about possible groundings of scene graphs to test images. The likelihoods of these groundings are used as ranking scores for retrieval. We introduce a novel dataset of 5,000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval. In particular, we evaluate retrieval using full scene graphs and small scene subgraphs, and show that our method outperforms retrieval methods that use only objects or low-level image features. In addition, we show that our full model can be used to improve object localization compared to baseline methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“Datasets have been a driving force of computer vision algorithms, ranging from classi-fication [64] to object recognition and segmentation [18, 12, 55, 15].”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“Their sys-tem is constrained to the six object classes from KITTI [23] while our system uses an open-world vocabulary; our scene graphs also tend to be more complex than their semantic graphs.”
MethodConditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Research works printed but not linked: 5
- jurafsky, 2000, Speech & language processing
- lampert, 2013, Attribute-based classification for zero-shot learning of object categories, PAMI
- yao, 2011, Classifying actions and measuring action similarity by modeling the mutual context of objects and human poses, a) A 1(D2) D3
- 0, OpenSceneGraph
- 10.1145/1964921.1964929 · DOI · Paper page
Cited by
Items 201–300 of 1,001
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning
Hessel, J., Hwang, J. D., Park, J. S., Zellers, R., Bhagavatula, C., Rohrbach, A., Saenko, K., & Choi, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_32 · Cited by 76
Target Adaptive Context Aggregation for Video Scene Graph Generation
Teng, Y., Wang, L., Li, Z., & Wu, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01343 · Cited by 74
Devil's on the Edges: Selective Quad Attention for Scene Graph Generation
Jung, D., Kim, S., Kim, W. H., & Cho, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01790 · Cited by 74
Learning to generalize to new compositions in image understandingPreprint
Atzmon, Y., Berant, J., Kezami, V., Globerson, A., & Chechik, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07639 · Cited by 74
Tensorize, Factorize and Regularize: Robust Visual Relationship Learning
Hwang, S. J., Kim, H. J., Ravi, S. N., Collins, M. D., Tao, Z., & Singh, V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00112 · Cited by 73
Crowdsourcing in Computer Vision
Kovashka, A., Russakovsky, O., Fei-Fei, L., & Grauman, K. (2016) · now publishers, Inc. eBooks · doi:10.1561/9781680832136 · Cited by 72
Self-citationHuang, R., Lin, H., Chen, C., Zhang, K., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642824 · Cited by 72
Onsite video mining for construction hazards identification with visual relationships
Xiong, R., Song, Y., Li, H., & Wang, Y. (2019) · Advanced Engineering Informatics · doi:10.1016/j.aei.2019.100966 · Cited by 72
Cross-Modal Hierarchical Modelling for Fine-Grained Sketch Based Image Retrieval
Sain, A., Bhunia, A. K., Yang, Y., Xiang, T., & Song, Y.-Z. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.28 · Cited by 72
Characterizing International Travel Behavior from Geotagged Photos: A Case Study of Flickr
Yuan, Y., & Medel, M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0154885 · Cited by 71
Scene Graph Prediction With Limited Labels
Krishna, R., Chen, V. S., Varma, P., Bernstein, M. S., Ré, C., & Fei-Fei, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00267 · Cited by 71
Self-citationA Graph-Based Framework to Bridge Movies and Synopses
Xiong, Y., Huang, Q., Guo, L., Zhou, H., Zhou, B., & Lin, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00469 · Cited by 71
Learning Common Sense through Visual Abstraction
Vedantam, R., Lin, X., Batra, T., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.292 · Cited by 70
Unified Graph Structured Models for Video Understanding
Arnab, A., Sun, C., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00801 · Cited by 70
FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsing
Li, Z., Chai, Y., Zhuo, T. Y., Qu, L., Haffari, G., Li, F., Ji, D., & Tran, Q. H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.398 · Cited by 69
SGFormer: Semantic Graph Transformer for Point Cloud-Based 3D Scene Graph Generation
Lv, C., Qi, M., Li, X., Yang, Z., & Ma, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28197 · Cited by 69
Adversarial Attribute-Text Embedding for Person Search With Natural Language Query
Zha, Z.-J., Liu, J., Chen, D., & Wu, F. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2972168 · Cited by 69
Image-to-Image Retrieval by Learning Similarity between Scene Graphs
Yoon, S., Kang, W. Y., Jeon, S., Lee, S., Han, C., Park, J., & Kim, E. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17281 · Cited by 68
Continuous Scene Representations for Embodied AI
Gadre, S. Y., Ehsani, K., Song, S., & Mottaghi, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01443 · Cited by 68
Unbiased Scene Graph Generation via Two-Stage Causal Modeling
Sun, S., Zhi, S., Liao, Q., Heikkilä, J., & Liu, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3285009 · Cited by 68
Scene Graph Parsing as Dependency Parsing
Wang, Y.-S., Liu, C., Zeng, X., & Yuille, A. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1037 · Cited by 68
Unsupervised Cross-Media Retrieval Using Domain Adaptation With Scene Graph
Peng, Y., & Chi, J. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2953692 · Cited by 68
Multi-interaction Network with Object Relation for Video Question Answering
Jin, W., Zhao, Z., Gu, M., Yu, J., Xiao, J., & Zhuang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351065 · Cited by 67
Deep Generative Probabilistic Graph Neural Networks for Scene Graph Generation
Khademi, M., & Schulte, O. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6783 · Cited by 67
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
Rodin, I., Furnari, A., Min, K., Tripathi, S., & Farinella, G. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01762 · Cited by 66
Context-Aware Embeddings for Automatic Art Analysis
García, N., Renoust, B., & Nakashima, Y. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325028 · Cited by 66
ContextNet: representation and exploration for painting classification and retrieval in context
García, N., Renoust, B., & Nakashima, Y. (2019) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-019-00189-4 · Cited by 66
Weakly Supervised Visual Semantic Parsing
Zareian, A., Karaman, S., & Chang, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00379 · Cited by 66
Crowdsourcing in Computer Vision
Kovashka, A., Russakovsky, O., Fei-Fei, L., & Grauman, K. (2016) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000071 · Cited by 65
Self-citationResistance Training Using Prior Bias: Toward Unbiased Scene Graph Generation
Chen, C., Zhan, Y., Yu, B., Liu, L., Luo, Y. H., & Du, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19896 · Cited by 65
Scene Graph Prediction with Limited Labels
Chen, V. S., Varma, P., Krishna, R., Bernstein, M. S., Ré, C., & Fei-Fei, L. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00220 · Cited by 64
Self-citationTIDEE: Tidying Up Novel Rooms Using Visuo-Semantic Commonsense Priors
Sarch, G. H., Fang, Z., Harley, A. W., Schydlo, P., Tarr, M. J., Gupta, S., & Fragkiadaki, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_28 · Cited by 64
A Causal Adjustment Module for Debiasing Scene Graph Generation
Liu, L., Sun, S., Zhi, S., Shi, F., Liu, Z., Heikkilä, J., & Liu, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3537283 · Cited by 64
BGT-Net: Bidirectional GRU Transformer Network for Scene Graph Generation
Dhingra, N., Ritter, F., & Kunz, A. M. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00244 · Cited by 64
(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering
Cherian, A., Hori, C., Marks, T. K., & Roux, J. L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19922 · Cited by 64
GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering
Liang, W., Jiang, Y., & Liu, Z. (2021) · doi:10.18653/v1/2021.maiworkshop-1.12 · Cited by 63
SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis
Farshad, A., Yeganeh, Y., Chi, Y., Shen, C., Ommer, B., & Navab, N. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00016 · Cited by 63
3DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understanding
Zemskova, T., & Yudin, D. A. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00831 · Cited by 62
Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation
Knyazev, B. A., de Vries, H., Cangea, C., Taylor, G. W., Courville, A. C., & Belilovsky, E. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.99 · Cited by 62
Linguistic Structures as Weak Supervision for Visual Scene Graph Generation
Ye, K., & Kovashka, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00819 · Cited by 62
A survey on generative adversarial network-based text-to-image synthesis
Zhou, R. H., Jiang, C. F., & Xu, Q. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.069 · Cited by 61
Integrating Knowledge and Reasoning in Image Understanding
Aditya, S., Yang, Y., & Baral, C. R. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/873 · Cited by 61
Visual Relationships as Functions:Enabling Few-Shot Scene Graph Prediction
Dornadula, A., Narcomey, A., Krishna, R., Bernstein, M. S., & Fei-Fei, L. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00214 · Cited by 61
Self-citationAn End-To-End Network for Generating Social Relationship Graphs
Goel, A., Ma, K. T., & Tan, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01144 · Cited by 61
Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos
Huang, D.-A., Lim, J. J., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.116 · Cited by 60
Self-citationFine-Grained Instance-Level Sketch-Based Image Retrieval
Yu, Q., Song, J., Song, Y.-Z., Xiang, T., & Hospedales, T. M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01382-3 · Cited by 60
Interactive augmented reality storytelling guided by scene semantics
Li, C., Li, W., Huang, H., & Yu, L.-F. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530061 · Cited by 59
Exploring Hierarchical Spatial Layout Cues for 3D Point Cloud Based Scene Graph Prediction
Feng, M., Hou, H., Zhang, L., Guo, Y., Yu, H., Wang, Y., & Mian, A. S. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3277736 · Cited by 59
On support relations and semantic scene graphs
Yang, M. Y., Liao, W., Ackermann, H., & Rosenhahn, B. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.07.010 · Cited by 59
Object-Centric Image Generation from Layouts
Sylvain, T., Zhang, P., Bengio, Y., Hjelm, R. D., & Sharma, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16368 · Cited by 58
Graph neural networks for image‐guided disease diagnosis: A review
Zhang, L., Zhao, Y., Che, T., Li, S., & Wang, X. (2023) · iRadiology · doi:10.1002/ird3.20 · Cited by 58
Deep Binaries: Encoding Semantic-Rich Cues for Efficient Textual-Visual Cross Retrieval
Shen, Y., Liu, L., Shao, L., & Song, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.441 · Cited by 58
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs
Herzig, R., Mendelson, A., Karlinsky, L., Arbelle, A., Feris, R., Darrell, T., & Globerson, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.870 · Cited by 57
Graph neural networks in vision-language image understanding: a survey
Senior, H., Slabaugh, G., Yuan, S., & Rossi, L. (2024) · The Visual Computer · doi:10.1007/s00371-024-03343-0 · Cited by 57
Detecting Human-Object Relationships in Videos
Ji, J., Desai, R., & Niebles, J. C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00800 · Cited by 57
Interactive Image Generation Using Scene GraphsPreprint
Mittal, G., Agrawal, S., Agarwal, A., Mehta, S., & Marwah, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03743 · Cited by 57
An Empirical Study on Leveraging Scene Graphs for Visual Question AnsweringPreprint
Zhang, C., Chao, W., & Xuan, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12133 · Cited by 57
DocTr: Document Transformer for Structured Information Extraction in Documents
Liao, H., RoyChowdhury, A., Li, W., Bansal, A., Zhang, Y., Tu, Z., Satzoda, R. K., Manmatha, R., & Mahadevan, V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01794 · Cited by 56
The Scene Language: Representing Scenes with Programs, Words, and Embeddings
Zhang, Y., Li, Z., Zhou, M., Wu, S., & Wu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02293 · Cited by 56
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation
Wang, W., Wang, R., Shan, S., & Chen, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_14 · Cited by 56
Interpretable Neural Subgraph Matching for Graph Retrieval
Roy, I., Velugoti, V. S. B. R., Chakrabarti, S., & De, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20784 · Cited by 55
Sim VQA: Exploring Simulated Environments for Visual Question Answering
Cascante-Bonilla, P., Wu, H., Wang, L., Feris, R., & Ordóñez, V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00500 · Cited by 55
Exploiting Long-Term Dependencies for Generating Dynamic Scene Graphs
Feng, S., Mostafa, H., Nassar, M., Majumdar, S., & Tripathi, S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00510 · Cited by 55
Graphhopper: Multi-hop Scene Graph Reasoning for Visual Question Answering
Koner, R., Li, H., Hildebrandt, M. E., Das, D., Tresp, V., & Günnemann, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88361-4_7 · Cited by 55
3D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphs
Looper, S., Rodríguez-Puigvert, J., Siegwart, R., Cadena, C., & Schmid, L. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161212 · Cited by 54
Al‐Shamayleh, A. S., Adwan, O., Alsharaiah, M. A., Hussein, A. H., Kharma, Q. M., & Eke, C. I. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18307-8 · Cited by 54
NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation
Li, L., Xiao, J., Shi, H., Zhang, H., Yang, Y., Liu, W., & Chen, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387349 · Cited by 54
Self-citationLiu, J., Meng, S., Gao, Y., Mao, S., Cai, P., Yan, G., Chen, Y., Bian, Z., Wang, D., & Shi, B. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00099 · Cited by 54
IS-GGT: Iterative Scene Graph Generation with Generative Transformers
Kundu, S., & Aakur, S. N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00609 · Cited by 52
Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation
Yan, X., Yuan, Z., Du, Y., Liao, Y., Guo, Y., Cui, S. R., & Li, Z. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3340679 · Cited by 52
Learning to Describe Scenes with Programs
Liu, Y., Wu, Z., Ritchie, D., Freeman, W. T., Tenenbaum, J. B., & Wu, J. (2018) · International Conference on Learning Representations · Cited by 52
Challenges and Prospects in Vision and Language Research
Kafle, K., Shrestha, R., & Kanan, C. (2019) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2019.00028 · Cited by 52
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
Wang, Z., You, H., Li, L. H., Zareian, A., Park, S., Liang, Y., Chang, K.-W., & Chang, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20536 · Cited by 52
Self-citationWhat Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation ModelsPreprint
Deng, T., Pan, Y., Yuan, S., Dong, L., Wang, C., Li, M., Chen, L., Xie, L., Wang, D., Wang, J., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.03422 · Cited by 52
T 2 SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Driving
Lv, C., Qi, M., Liu, L., & Ma, H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01603 · Cited by 51
Weakly-Supervised Image Annotation and Segmentation with Objects and Attributes
Shi, Z., Yang, Y., Hospedales, T. M., & Xiang, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2645157 · Cited by 51
On Diversity in Image Captioning: Metrics and Methods
Wang, Q., Wan, J., & Chan, A. B. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3013834 · Cited by 51
Interpretable and Globally Optimal Prediction for Textual Grounding using Image ConceptsPreprint
Yeh, R. A., Xiong, J., Hwu, W., Do, M. N., & Schwing, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11209 · Cited by 51
Re-evaluating Automatic Metrics for Image CaptioningPreprint
Kilickaya, M., Erdem, A., İkizler-Cinbiş, N., & Erdem, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07600 · Cited by 51
Detecting Human-Object Interactions with Object-Guided Cross-Modal Calibrated Semantics
Yuan, H., Wang, M., Ni, D., & Xu, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20229 · Cited by 50
HyperSOR: Context-Aware Graph Hypernetwork for Salient Object Ranking
Qiao, M., Xu, M., Jiang, L. W., Lei, P., Wen, S., Chen, Y., & Sigal, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3368158 · Cited by 50
Visual Graphs from Motion (VGfM): Scene Understanding with Object Geometry Reasoning
Gay, P., James, S., & Bue, A. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_21 · Cited by 50
Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphs
Gao, K., Chen, L., Niu, Y., Shao, J., & Xiao, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01889 · Cited by 50
Dual-Branch Hybrid Learning Network for Unbiased Scene Graph Generation
Zheng, C., Gao, L., Lyu, X., Zeng, P., Saddik, A. E., & Shen, H. T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3297842 · Cited by 49
Learning From Interaction-Enhanced Scene Graph for Pedestrian Collision Risk Assessment
Liu, X., Zhou, Y., & Gou, C. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3309274 · Cited by 49
Özsoy, E., Pellegrini, C., Czempiel, T., Tristram, F., Yuan, K., Bani-Harouni, D., Eck, U., Busam, B., Keicher, M., & Navab, N. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01805 · Cited by 49
Text-Guided Neural Network Training for Image Recognition in Natural Scenes and Medicine
Zhang, Z., Chen, P., Shi, X., & Yang, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2955476 · Cited by 49
Visual Distant Supervision for Scene Graph Generation
Yao, Y., Zhang, A., Han, X., Li, M., Weber, C., Liu, Z., Wermter, S., & Sun, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01552 · Cited by 49
Auto-Parsing Network for Image Captioning and Visual Question Answering
Yang, X., Gao, C., Zhang, H., & Cai, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00220 · Cited by 49
From Images to Sentences through Scene Description Graphs using Commonsense Reasoning and Knowledge
Aditya, S., Yang, Y., Baral, C. R., Fermüller, C., & Aloimonos, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03292 · Cited by 48
SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstruction
Koch, S., Hermosilla, P., Vaškevičius, N., Colosi, M., & Ropinski, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00337 · Cited by 48
Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation
Wang, J., Wen, Z., Li, X., Guo, Z., Yang, J., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3442301 · Cited by 48
HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation
Zhang, C., Stepputtis, S., Campbell, J., Sycara, K. P., & Xie, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02667 · Cited by 48
Grounding Predicates through Actions
Migimatsu, T., & Bohg, J. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812016 · Cited by 48
MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENeration
Hayes, T., Zhang, S., Yin, X. P., Pang, G., Sheng, S., Yang, H. C., Ge, S., Hu, Q., & Parikh, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_25 · Cited by 48
Learning hierarchical relationships for object-goal navigationPreprint
Qiu, Y., Pal, A., & Christensen, H. I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06749 · Cited by 48
SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs
Zhai, G., Cai, X., Huang, D., Di, Y., Manhardt, F., Tombari, F., Navab, N., & Busam, B. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610792 · Cited by 47
Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation
Li, L., Xiao, J., Shi, H., Wang, W., Shao, J., Liu, A.-A., Yang, Y., & Chen, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3282349 · Cited by 47
Self-citationRethinking Benchmarks for Cross-modal Image-text Retrieval
Chen, W., Yao, L., & Jin, Q. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591758 · Cited by 46
Visual knowledge in the big model era: retrospect and prospect
Wang, W., Yang, Y., & Pan, Y. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2400250 · Cited by 46
Metadata
- Authors
- Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Max Planck Institute for Informatics (DE)
- Yahoo (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3668-3678
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2077069816
- Cited by
- 1,101
- References
- 68
- Citations by year
- 2026: 59 · 2025: 84 · 2024: 133 · 2023: 134 · 2022: 130 · 2021: 139 · 2020: 145 · 2019: 117 · 2018: 66 · 2017: 47 · 2016: 33 · 2015: 13
- FWCI
- 21.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- semantic image retrieval, scene graph, conditional random fields, object relations, image ranking, object localization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.