The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation
Open accessAbstract
Understanding the visual relationship between two objects involves identifying the subject, the object, and a predicate relating them. We leverage the strong correlations between the predicate and the hsubj; obji pair (both semantically and spatially) to predict predicates conditioned on the subjects and the objects. Modeling the three entities jointly more accurately reflects their relationships compared to modeling them independently, but it complicates learning since the semantic space of visual relationships is huge and training data is limited, especially for longtail relationships that have few instances. To overcome this, we use knowledge of linguistic statistics to regularize visual model learning. We obtain linguistic knowledge by mining from both training annotations (internal knowledge) and publicly available text, e.g., Wikipedia (external knowledge), computing the conditional probability distribution of a predicate given a (subj, obj) pair. As we train the visual model, we distill this knowledge into the deep model to achieve better generalization. Our experimental results on the Visual Relationship Detection (VRD) and Visual Genome datasets suggest that with this linguistic knowledge distillation, our model outperforms the stateof- the-art methods significantly, especially when predicting unseen relationships (e.g., recall improved from 8.45% to 19.17% on VRD zero-shot testing set).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Visual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,069
Dependency tree kernels for relation extraction
Culotta, A., & Sorensen, J. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1219009 · Cited by 814
Exploring various knowledge in relation extraction
GuoDong, Z., Jian, S., Jie, Z., & Min, Z. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219893 · Cited by 764
Harnessing Deep Neural Networks with Logic Rules
Hu, Z., Ma, X., Liu, Z., Hovy, E., & Xing, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1228 · Cited by 545
Detecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.352 · Cited by 502
Object categorization using co-occurrence, location and appearance
Galleguillos, C., Rabinovich, A., & Belongie, S. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587799 · Cited by 457
Modeling Context Between Objects for Referring Expression Understanding
Nagaraja, V. K., Morariu, V. I., & Davis, L. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_48 · Cited by 425
Self-citationRecognition using visual phrases
Sadeghi, M. A., & Farhadi, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995711 · Cited by 417
Context based object categorization: A critical survey
Galleguillos, C., & Belongie, S. (2010) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.02.004 · Cited by 351
Large-Scale Object Classification Using Label Relation Graphs
Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., & Adam, H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_4 · Cited by 347
Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval
Schuster, S., Krishna, R., Chang, A., Fei-Fei, L., & Manning, C. D. (2015) · Proceedings of the Fourth Workshop on Vision and Language · doi:10.18653/v1/w15-2812 · Cited by 332
Learning to share visual appearance for multiclass object detection
Salakhutdinov, R., Torralba, A., & Tenenbaum, J. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995720 · Cited by 325
Graph Cut Based Inference with Co-occurrence Statistics
Ladicky, L., Russell, C., Kohli, P., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_18 · Cited by 308
Deep Variation-Structured Reinforcement Learning for Visual Relationship and Attribute Detection
Liang, X., Lee, L., & Xing, E. P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.469 · Cited by 243
When Are Tree Structures Necessary for Deep Learning of Representations?
Li, J., Luong, T., Jurafsky, D., & Hovy, E. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1278 · Cited by 207
COSTA: Co-Occurrence Statistics for Zero-Shot Classification
Mensink, T., Gavves, E., & Snoek, C. G. M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.313 · Cited by 193
Tree Kernel-Based Relation Extraction with Context-Sensitive Structured Parse Tree Information
Zhou, G., Zhang, M., Ji, D., & Zhu, Q. (2007) · Cited by 189
Deep Neural Networks with Massive Learned Knowledge
Hu, Z., Yang, Z., Salakhutdinov, R., & Xing, E. P. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1173 · Cited by 82
The Role of Context Selection in Object Detection
Yu, R., Chen, X., Morariu, V. I., & Davis, L. S. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.133 · Cited by 40
Self-citationGenerating Holistic 3D Scene Abstractions for Text-Based Image Retrieval
Li, A., Sun, J., Ng, J. Y.-H., Yu, R., Morariu, V. I., & Davis, L. S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.210 · Cited by 22
Self-citationComposing graphical models with neural networks for structured representations and fast inference
Johnson, M. J., Duvenaud, D., Wiltschko, A. B., Datta, S. R., & Adams, R. P. (2016) · arXiv · doi:10.48550/arxiv.1603.06277
Cited by
Improved Knowledge Distillation via Teacher Assistant
Mirzadeh, S. I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., & Ghasemzadeh, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5963 · Cited by 101
“To solve this issue, Yim et al.; Yu et al. (2017; 2017) used a shared representation of layers, however, it’s not straight-forward to choose the appropriate layer to be matched.”
BackgroundSelf-citationNeural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00611 · Cited by 1,052
Revisiting Knowledge Distillation via Label Smoothing Regularization
Yuan, L., Tay, F. E., Li, G., Wang, T., & Feng, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00396 · Cited by 572
A Comprehensive Survey of Scene Graphs: Generation and Application
Chang, X., Ren, P., Xu, P., Li, Z., Chen, X., & Hauptmann, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3137605 · Cited by 324
Scene Graph Generation With External Knowledge and Image Reconstruction
Gu, J., Zhao, H., Lin, Z., Li, S., Cai, J., & Ling, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00207 · Cited by 308
Metadata
- Authors
- Yu, R., Li, A., Morariu, V. I., & Davis, L. S.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1068-1076
- OpenAlex
- W2962737704
- Cited by
- 310
- References
- 29
- FWCI
- 11.24
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.