The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Knowledge-Embedded Routing Network for Scene Graph Generation
Open accessAbstract
To understand a scene in depth not only involves locating/recognizing individual objects, but also requires to infer the relationships and interactions among them. However, since the distribution of real-world relationships is seriously unbalanced, existing methods perform quite poorly for the less frequent relationships. In this work, we find that the statistical correlations between object pairs and their relationships can effectively regularize semantic space and make prediction less ambiguous, and thus well address the unbalanced distribution issue. To achieve this, we incorporate these statistical correlations into deep neural networks to facilitate scene graph generation by developing a Knowledge-Embedded Routing Network. More specifically, we show that the statistical correlations between objects appearing in images and their relationships, can be explicitly represented by a structured knowledge graph, and a routing mechanism is learned to propagate messages through the graph to explore their interactions. Extensive experiments on the large-scale Visual Genome dataset demonstrate the superiority of the proposed method over current state-of-the-art competitors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 2×
“[20] constructed a knowledge graph based on the WordNet [22] and the Visual Genome dataset [14], and learned the representation of this graph to enhance image feature representation to promote multilabel recognition.”
MethodThe graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Scene Graph Generation by Iterative Message Passing
Xu, D., Zhu, Y., Choy, C. B., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.330 · Cited by 1,271
Image retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101
Visual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,069
Neural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00611 · Cited by 1,052
Learning to Detect Human-Object Interactions
Chao, Y.-W., Liu, Y., Liu, X., Zeng, H., & Deng, J. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00048 · Cited by 520
Detecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.352 · Cited by 502
Scene Graph Generation from Objects, Phrases and Region Captions
Li, Y., Ouyang, W., Zhou, B., Wang, K., & Wang, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.142 · Cited by 500
Object categorization using co-occurrence, location and appearance
Galleguillos, C., Rabinovich, A., & Belongie, S. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587799 · Cited by 457
Multi-Class Segmentation with Relative Location Prior
Gould, S., Rodgers, J., Cohen, D., Elidan, G., & Koller, D. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0140-x · Cited by 400
Large-Scale Object Classification Using Label Relation Graphs
Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., & Adam, H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_4 · Cited by 347
Grouplet: A structured image representation for recognizing human and object interactions
Yao, B., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540234 · Cited by 328
The More You Know: Using Knowledge Graphs for Image Classification
Marino, K., Salakhutdinov, R., & Gupta, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.10 · Cited by 315
Multi-label Zero-Shot Learning with Structured Knowledge Graphs
Lee, C.-W., Fang, W., Yeh, C.-K., & Wang, Y.-C. F. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00170 · Cited by 305
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
Yatskar, M., Zettlemoyer, L., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.597 · Cited by 242
Pixels to Graphs by Associative Embedding
Newell, A., & Deng, J. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.07365 · Cited by 178
Deep Reasoning with Knowledge Graph for Social Relationship Understanding
Wang, Z., Chen, T., Ren, J., Yu, W., Cheng, H., & Lin, L. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/142 · Cited by 178
Self-citationUnderstanding Indoor Scenes Using 3D Geometric Phrases
Choi, W., Chao, Y.-W., Pantofaru, C., & Savarese, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.12 · Cited by 176
Learning the Visual Interpretation of Sentences
Zitnick, C. L., Parikh, D., & Vanderwende, L. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.211 · Cited by 152
Object Detection Meets Knowledge Graphs
Fang, Y., Kuan, K., Lin, J., Tan, C., & Chandrasekhar, V. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/230 · Cited by 128
Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition
Chen, T., Lin, L., Chen, R., Yang, W., & Luo, X. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/87 · Cited by 98
Self-citationNeural Task Planning With AND–OR Graph Representations
Chen, T., Chen, R., Nie, L., Luo, X., Liu, X., & Lin, L. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2870062 · Cited by 17
Self-citationKnowledge-guided recurrent neural network learning for task-oriented action prediction
Lin, L., Huang, L., Chen, T., Gan, Y., & Cheng, H. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019345 · Cited by 16
Self-citation
Cited by
Unbiased Scene Graph Generation From Biased Training
Tang, K., Niu, Y., Huang, J., Shi, J., & Zhang, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00377 · Cited by 755
Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
Chen, T., Xu, M., Hui, X., Wu, H., & Lin, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00061 · Cited by 332
Self-citationA Comprehensive Survey of Scene Graphs: Generation and Application
Chang, X., Ren, P., Xu, P., Li, Z., Chen, X., & Hauptmann, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3137605 · Cited by 324
GPS-Net: Graph Property Sensing Network for Scene Graph Generation
Lin, X., Ding, C., Zeng, J., & Tao, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00380 · Cited by 287
From General to Specific: Informative Scene Graph Generation via Balance Adjustment
Guo, Y., Gao, L., Wang, X., Hu, Y., Xu, X., Lu, X., Shen, H. T., & Song, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01607 · Cited by 97
Metadata
- Authors
- Chen, T., Yu, W., Chen, R., & Lin, L.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 6156-6164
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2962785943
- Cited by
- 432
- References
- 34
- FWCI
- 18.98
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.