Image retrieval using scene graphs
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
This paper develops a novel framework for semantic image retrieval based on the notion of a scene graph. Our scene graphs represent objects (“man”, “boat”), attributes of objects (“boat is white”) and relationships between objects (“man standing on boat”). We use these scene graphs as queries to retrieve semantically related images. To this end, we design a conditional random field model that reasons about possible groundings of scene graphs to test images. The likelihoods of these groundings are used as ranking scores for retrieval. We introduce a novel dataset of 5,000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval. In particular, we evaluate retrieval using full scene graphs and small scene subgraphs, and show that our method outperforms retrieval methods that use only objects or low-level image features. In addition, we show that our full model can be used to improve object localization compared to baseline methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“Datasets have been a driving force of computer vision algorithms, ranging from classi-fication [64] to object recognition and segmentation [18, 12, 55, 15].”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“Their sys-tem is constrained to the six object classes from KITTI [23] while our system uses an open-world vocabulary; our scene graphs also tend to be more complex than their semantic graphs.”
MethodConditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Research works printed but not linked: 5
- jurafsky, 2000, Speech & language processing
- lampert, 2013, Attribute-based classification for zero-shot learning of object categories, PAMI
- yao, 2011, Classifying actions and measuring action similarity by modeling the mutual context of objects and human poses, a) A 1(D2) D3
- 0, OpenSceneGraph
- 10.1145/1964921.1964929 · DOI · Paper page
Cited by
Items 701–800 of 1,001
Multimodal graph inference network for scene graph generation
Duan, J., Min, W., Lin, D., Xu, J., & Xiong, X. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02304-7 · Cited by 9
Extracting Structured Supervision From Captions for Weakly Supervised Semantic Segmentation
Vilar, D. R., & Perez, C. A. (2021) · IEEE Access · doi:10.1109/access.2021.3076074 · Cited by 9
Aligning vision-language for graph inference in visual dialog
Jiang, T., Shao, H., Tian, X., Ji, Y., & Liu, C. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104316 · Cited by 9
Residual objectness for imbalance reduction
Chen, J., Liu, D., Luo, B., Peng, X., Xu, T., & Chen, E. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108781 · Cited by 9
Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical ProceduresPreprint
Özsoy, E., Örnek, E. P., Eck, U., Tombari, F., & Navab, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15309 · Cited by 9
Scene Graph Generation for Better Image Captioning?Preprint
Mozes, M., Schmitt, M., Golkov, V., Schütze, H., & Cremers, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11398 · Cited by 9
Relationship Oriented Affordance Learning through Manipulation Graph ConstructionPreprint
Tang, C., Yu, J., Chen, W., & Zhang, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14137 · Cited by 9
Multi-modal reward for visual relationships-based image captioningPreprint
Abedi, A., Karshenas, H., & Adibi, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.10766 · Cited by 9
AVR: Attention based Salient Visual Relationship DetectionPreprint
Lv, J., Xiao, Q., & Zhong, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07012 · Cited by 9
Searching in Space and Time: Unified Memory-Action Loops for Open-World Object Retrieval
Chen, T., Kumar, S., Xu, J., Pavlakos, G., Biswas, J., & Martín-Martín, R. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696368 · Cited by 8
SciFlow-Bench: Evaluating Structure-Aware Scientific Diagram Generation via Inverse Parsing
Zhang, T., Lin, H., Liu, Z., Chen, C., & Zhang, W. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.807 · Cited by 8
Unbiased 3D Semantic Scene Graph Prediction in Point Cloud Using Deep Learning
Han, C., Li, H., Xu, J. G., Dong, B., Wang, Y., Zhou, X., & Zhao, S. (2023) · Applied Sciences · doi:10.3390/app13095657 · Cited by 8
VR+HD: Video Semantic Reconstruction From Spatio-Temporal Scene Graphs
Li, C., Duan, Y., Du, Q., Sun, S., Deng, X., & Tao, X. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3323654 · Cited by 8
JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models
Wada, Y., Kaneda, K., & Sugiura, K. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.28 · Cited by 8
Gaussian Distribution-Aware Commonsense Knowledge Learning for Scene Graph Generation
Tian, H., Xu, N., Kankanhalli, M., & Liu, A.-A. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3443417 · Cited by 8
Yuan, W., Chang, D., Mao, C., Wang, L., Ren, K. L., & Han, T. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103161 · Cited by 8
Semi-Supervised Clustering Framework for Fine-grained Scene Graph Generation
Yang, J., Wang, C., Zhang, J., Wu, S., Zhao, J., Liu, Z., & Yang, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i9.32998 · Cited by 8
Union-Redefined Prototype Network for scene graph generation
Jung, N., & Choi, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127486 · Cited by 8
Unintended Bias Detection and Mitigation in Misogynous Memes
Kumari, G., Sinha, A., & Ekbal, A. (2024) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2024.eacl-long.166 · Cited by 8
BACON: Improving Clarity of Image Captions via Bag-of-Concept Graphs
Yang, Z., Feng, R., Yan, K., Wang, H., Wang, Z., Zhu, S., Zhang, H., Xiao, J., Wu, P., Zhu, K., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01341 · Cited by 8
Dual Scene Graph Convolutional Network for Motivation Prediction
Wanyan, Y., Yang, X., Ma, X., & Xu, C. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3572914 · Cited by 8
Semantic Image Collection Summarization With Frequent Subgraph Mining
Pasini, A., Giobergia, F., Pastor, E., & Baralis, E. (2022) · IEEE Access · doi:10.1109/access.2022.3229654 · Cited by 8
Relationship Spatialization for Depth Estimation
Xu, X., Qiu, J., Wang, X., & Wang, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_35 · Cited by 8
Synthesizing Play-Ready VR Scenes with Natural Language Prompts Through GPT API
Kouzelis, L. R., & Spantidi, O. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47966-3_2 · Cited by 8
Commonsense Knowledge-Driven Joint Reasoning Approach for Object Retrieval in Virtual Reality
Jiang, H., Weng, D., Dongye, X., Luo, L., & Zhang, Z. (2023) · ACM Transactions on Graphics · doi:10.1145/3618320 · Cited by 8
Fine-Grained is Too Coarse: A Novel Data-Centric Approach for Efficient Scene Graph Generation
Neau, M., Santos, P. E., Bosser, A.-G., & Buche, C. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00008 · Cited by 8
SPAN: Learning Similarity Between Scene Graphs and Images With Transformers
Cong, Y., Liao, W., Rosenhahn, B., & Yang, M. Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3630242 · Cited by 8
Semantic Tuples for Evaluation of Image to Sentence Generation
Ellebracht, L. D., Ramisa, A., Madhyastha, P., Cordero-Rama, J., Moreno-Noguer, F., & Quattoni, A. (2015) · Workshop on Vision and Language (VL) · doi:10.18653/v1/w15-2806 · Cited by 8
Multimodal clothing recognition for semantic search in unconstrained surveillance imagery
Halstead, M., Denman, S., Sridharan, S., Tian, Y., & Fookes, C. (2018) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.12.001 · Cited by 8
GAL: A global-attributes assisted labeling system for outdoor scenes
Ren, Y., Chen, C., Li, S., & Kuo, C. ‐. J. (2016) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2016.11.004 · Cited by 8
Wu, R., Xu, K., Liu, C., Zhuang, N., & Mu, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6913 · Cited by 8
Attentive Gated Graph Neural Network for Image Scene Graph Generation
Li, S., Tang, M., Zhang, J., & Jiang, L. (2020) · Symmetry · doi:10.3390/sym12040511 · Cited by 8
Graph-Based Indexing and Retrieval of Lifelog Data
Nguyen, M.-D., Nguyen, B. T., & Gurrin, C. G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67835-7_22 · Cited by 8
Scene graph generation by multi-level semantic tasks
Tian, P., Mo, H., & Jiang, L. (2021) · Applied Intelligence · doi:10.1007/s10489-020-02115-2 · Cited by 8
Graph-Based Information Block Detection in Infographic With Gestalt Organization Principles
Lin, J., Cai, Y., Wu, X., & Lu, J. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3130071 · Cited by 8
Towards Open-Set Scene Graph Generation With Unknown Objects
Sonogashira, M., Iiyama, M., & Kawanishi, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3145465 · Cited by 8
High-Quality Image Generation from Scene Graphs with Transformer
Zhao, X., Wu, L., Chen, X., & Gong, B. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859841 · Cited by 8
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining
Bugliarello, E., Nematzadeh, A., & Hendricks, L. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.184 · Cited by 8
A Bottom-up Paradigm for Traffic Scene Graph Representation
Zhang, Z., Zhang, C., Liu, Y., Su, Y., Li, P., & Zheng, J. (2020) · International Conference on Computing and Pattern Recognition (ICCPR) · doi:10.1145/3436369.3437437 · Cited by 8
Scene Graph Modification Based on Natural Language Commands
He, X., Tran, Q. H., Haffari, G., Chang, W. H., Lin, Z., Bui, T., Dernoncourt, F., & Dam, N. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.87 · Cited by 8
Describing Natural Images Containing Novel Objects with Knowledge Guided AssitancePreprint
Mogadala, A., Bista, U., Xie, L., & Rettinger, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06303 · Cited by 8
Rethinking Visual Relationships for High-level Image Understanding.Preprint
Liang, Y., Bai, Y., Zhang, W., Qian, X., Zhu, L., & Mei, T. (2019) · arXiv (Cornell University) · Cited by 8
The Tensor Brain: Semantic Decoding for Perception and MemoryPreprint
Tresp, V., Sharifzadeh, S., Konopatzki, D., & Ma, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11027 · Cited by 8
Complex Scene Image Editing by Scene Graph ComprehensionPreprint
Zhang, Z., He, H., Plummer, B. A., Liao, Z., & Wang, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.12849 · Cited by 8
Symbolic image detection using scene and knowledge graphsPreprint
Kalanat, N., & Kovashka, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.04863 · Cited by 8
Knowledge-augmented Few-shot Visual Relation DetectionPreprint
Yu, T., Li, Y., Chen, J., Li, Y., Zheng, H.-T., Chen, X., Liu, Q., Liu, W., Huang, D., Wu, B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.05342 · Cited by 8
What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable GenerationPreprint
Chen, Z., Wu, J., Lei, Z., & Chen, C. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.15435 · Cited by 8
From Query to Explanation: Uni-RAG for Multi-Modal Retrieval-Augmented Learning in STEMPreprint
Wu, X., Jia, Y., Xiao, L., Zhao, S., Chiang, F., & Cambria, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.03868 · Cited by 8
NODIS: Neural Ordinary Differential Scene UnderstandingPreprint
Yuren, C., Ackermann, H., Liao, W., Yang, M. Y., & Rosenhahn, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04735 · Cited by 8
Decoding the surgical scene: A scoping review of scene graphs in surgery
Henriques, A., Hoxha, K., Zapp, D., Issa, P. C., Navab, N., & Nasseri, M. A. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104083 · Cited by 7
RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation
Yue, C., Chen, R., Zhang, Z., Chen, Y., Tian, Y., & Xie, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37350-2_13 · Cited by 7
Zhang, R., An, G., Cen, Y., & Ruan, Q. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111039 · Cited by 7
Mastering Scene Understanding: Scene Graphs to the Rescue
Caetano, C., Ribeiro, L. S. F., Laranjeira, C., dos Santos, G. O., Barros, A., Petrucci, C., dos Santos, A. A., Macedo, J., Carvalho, G., Benevenuto, F., et al. (2024) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi62404.2024.10716310 · Cited by 7
Calibration for Long-tailed Scene Graph Generation
Zhu, X., Xing, Y., Wang, R., Wang, Y., & Lan, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680818 · Cited by 7
NavTr: Object-Goal Navigation With Learnable Transformer Queries
Mao, Q., Wang, J., Xu, M., & Chen, Z. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3497718 · Cited by 7
Sun, S., Zeng, Z., Li, Q., & Li, T. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113126 · Cited by 7
Explicitly Guided Difficulty-Controllable Visual Question Generation
Xie, J., Cheng, M., Zhang, X., Cai, Y., Hu, G., Xie, M., & Li, Q. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i24.34745 · Cited by 7
Towards better text image machine translation with multimodal codebook and multi-stage training
Lan, Z., Yu, J., Liu, S., Yao, J., Huang, D., & Su, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107599 · Cited by 7
Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models
Yoon, E., Yoon, H. S., Hasegawa‐Johnson, M., & Yoo, C. D. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2507.04976 · Cited by 7
A Bottom-up Framework for Construction of Structured Semantic 3D Scene Graph
Yu, B., Chen, C., Zhou, F., Wan, F., Zhuang, W., & Zhao, Y. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341228 · Cited by 7
Improving rare relation inferring for scene graph generation using bipartite graph network
Lu, J., Chen, L., Guan, H., Lin, S., Gu, C., Wang, C., & He, G. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103901 · Cited by 7
CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation
Basioti, K., Abdelsalam, M. A., Fancellu, F., Pavlović, V., & Fazly, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72848-8_26 · Cited by 7
Hierarchical layered multigraph network with scale importance estimation for image classification
Batisteli, J. P. O., Passat, N., Guimarães, S. J. F., & Patrocínio, Z. K. G. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113877 · Cited by 7
Finding Weather Photos: Community-Supervised Methods for Editorial Curation of Online Sources
Shamma, D. A., Kennedy, L. S., Li, L.-J., Thomée, B., Jin, H., & Yuan, J. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2819989 · Cited by 7
Self-citationImproved Image Description Via Embedded Object Structure Graph and Semantic Feature Matching
Ren, L., & Hua, K. A. (2018) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism.2018.00021 · Cited by 7
MoQA – A Multi-modal Question Answering Architecture
Haurilet, M., Al-Halah, Z., & Stiefelhagen, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_9 · Cited by 7
Conser, E., Hahn, K., Watson, C. M., & Mitchell, M. (2019) · Proceedings of the Second Workshop on Shortcomings in Vision and Language · doi:10.18653/v1/w19-1804 · Cited by 7
Improving Trust in Deep Neural Networks with Nearest Neighbors
Lee, R., Clarke, J., Agogino, A., & Giannakopoulou, D. (2020) · AIAA SciTech Forum · doi:10.2514/6.2020-2098 · Cited by 7
Deep Neural Matching Models for Graph Retrieval
Goyal, K., Gupta, U., De, A., & Chakrabarti, S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401216 · Cited by 7
High-quality Activity-Level Video Advertising
Yuan, M., Zhang, L., Wu, Z., & Zheng, D. (2020) · International Workshop on Quality of Service · doi:10.1109/iwqos49365.2020.9212956 · Cited by 7
Visualizing Children Stories with Generated Image Sequences
Zakraoui, J., Saleh, M., Al-Maadeed, S. A., Alja’am, J. M., & El-Seoud, S. A. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-67209-6_55 · Cited by 7
Relationship Oriented Semantic Scene Understanding for Daily Manipulation Tasks
Tang, C., Yu, J., Chen, W., Xia, B., & Zhang, H. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981960 · Cited by 7
Attention-Based Scene Graph Generation: A Review
Airin, A., Dawla, R. U., Noor, A. S., Hasan, M. A., Hasan, A. R., Zaman, A., & Farid, D. M. (2022) · International Conference on Software, Knowledge, Information Management and Applications (SKIMA) · doi:10.1109/skima57145.2022.10029570 · Cited by 7
Multi-Granularity Semantic Information Integration Graph for Cross-Modal Hash Retrieval
Han, Z., Azman, A., Khalid, F., & Mustaffa, M. R. (2024) · IEEE Access · doi:10.1109/access.2024.3380019 · Cited by 7
Arulmozhi, P., & Abirami, S. (2017) · Artificial Intelligence Review · doi:10.1007/s10462-017-9591-1 · Cited by 7
Going Deeper With Semantics: Video Activity Interpretation Using Semantic Contextualization
Aakur, S. N., de Souza, F. D., & Sarkar, S. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00026 · Cited by 7
Visual-GPS: Ego-Downward and Ambient Video Based Person Location Association
Yang, L., Jiang, H., Huo, Z., & Xiao, J. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00050 · Cited by 7
Visual Relationship Detection with Contextual Information
Wang, Y. L. Y., & Zhu, Y. (2020) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2020.07451 · Cited by 7
UniVSE: Robust Visual Semantic Embeddings via Structured Semantic RepresentationsPreprint
Wu, H., Mao, J., Zhang, Y., Jiang, Y., Li, L., Sun, W., & Ma, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05521 · Cited by 7
Self-citationSemantic Image Manipulation Using Scene GraphsPreprint
Dhamo, H., Farshad, A., Laina, I., Navab, N., Hager, G. D., Tombari, F., & Rupprecht, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03677 · Cited by 7
Incremental Object Grounding Using Scene GraphsPreprint
Yi, J. S. K., Kim, Y., & Chernova, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.01901 · Cited by 7
SGRAM: Improving Scene Graph Parsing via Abstract Meaning RepresentationPreprint
Choi, W. S., Heo, Y., & Zhang, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.08675 · Cited by 7
DisPositioNet: Disentangled Pose and Identity in Semantic Image ManipulationPreprint
Farshad, A., Yeganeh, Y., Dhamo, H., Tombari, F., & Navab, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.05499 · Cited by 7
Detecting unseen visual relations using analogiesPreprint
Peyre, J., Laptev, I., Schmid, C., & Šivic, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05736 · Cited by 7
Seeing Beyond the Scene: Enhancing Vision-Language Models with Interactional ReasoningPreprint
Liang, D., Zheng, C., Wen, Z., Cai, Y., Wei, X.-Y., & Li, Q. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.09118 · Cited by 7
A$^2$RD: Agentic Autoregressive Diffusion for Long Video ConsistencyPreprint
Long, D. X., Song, Y., Kan, M., Pfister, T., & Le, L. T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.06924 · Cited by 7
Sherlock: Scalable Fact Learning in Images
Elhoseiny, M., Cohen, S. D., Chang, W. H., Price, B. L., & Elgammal, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11214 · Cited by 6
A New Training Data Organization Form and Training Mode for Unbiased Scene Graph Generation
Xu, H., Wang, L., Xu, K., Fu, F., Yin, B., & Huang, Q. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3344569 · Cited by 6
A Parallel Transformer Framework for Video Moment Retrieval
Nguyen, T.-N., Li, Z., Yamazaki, S., Liu, J., & Gurrin, C. G. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658096 · Cited by 6
Exploiting Visual Relation and Multi-Grained Knowledge for Multimodal Relation Extraction
Shen, Q., Lin, H., Liu, H., Zheng, L. L., & Wang, W. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650770 · Cited by 6
Chikwendu, I. A., Zhang, X., Monday, H. N., Nneji, G. U., Ukwuoma, C. C., Chikwendu, O. C., Gu, Y. H., & Al–antari, M. A. (2025) · Electronics · doi:10.3390/electronics14061158 · Cited by 6
Künzel, S., Munz, T., Tilli, P., Schäfer, N., Vidyapu, S., Vu, N. T., & Weiskopf, D. (2025) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-025-00185-y · Cited by 6
Yan, H., & Mu, Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i14.33613 · Cited by 6
ReCon1M: A Large-Scale Benchmark Dataset for Relation Comprehension in Remote Sensing Imagery
Yan, Q., Deng, C., Liu, C., Hou, Z., Liu, X., Jiang, Y., Lu, W., Yao, F., Liu, X., Hao, L., et al. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3589986 · Cited by 6
Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval
Li, S., Gao, X., & Du, S. S. (2025) · neural information processing systems · doi:10.52202/085713-3837 · Cited by 6
DiscoSG: Towards Discourse-Level Text Scene Graph Parsing through Iterative Graph Refinement
Lin, S., Teng, C., Li, F., Ji, D., Qu, L., & Li, Z. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.398 · Cited by 6
Kim, S., Joo, K.-N., & Youn, C. (2021) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc52510.2021.9621009 · Cited by 6
A Pipeline for Story Visualization from Natural Language
Zakraoui, J., Saleh, M., Al-Maadeed, S. A., & Alja’am, J. M. (2023) · Applied Sciences · doi:10.3390/app13085107 · Cited by 6
Wang, L., Yuan, Z., & Chen, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73007-8_7 · Cited by 6
Hypergraph-Guided Multimodal Prototype for Remote Sensing Scene Understanding
Liu, C., Deng, C., Yu, H., Yan, Q., Xu, L., Zhang, T., Sun, X., & Fu, K. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3534288 · Cited by 6
Metadata
- Authors
- Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Max Planck Institute for Informatics (DE)
- Yahoo (United States) (US)
- ORCID
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3668-3678
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2077069816
- Cited by
- 1,101
- References
- 68
- Citations by year
- 2026: 59 · 2025: 84 · 2024: 133 · 2023: 134 · 2022: 130 · 2021: 139 · 2020: 145 · 2019: 117 · 2018: 66 · 2017: 47 · 2016: 33 · 2015: 13
- FWCI
- 21.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Max Planck Digital Library (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- semantic image retrieval, scene graph, conditional random fields, object relations, image ranking, object localization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.