The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Scene Graph Generation from Objects, Phrases and Region Captions
Open accessAbstract
Object detection, scene graph generation and region captioning, which are three scene understanding tasks at different semantic levels, are tied together: scene graphs are generated on top of objects detected in an image with their pairwise relationship predicted, while region captioning gives a language description of the objects, their attributes, relations and other context information. In this work, to leverage the mutual connections across semantic levels, we propose a novel neural network model, termed as Multi-level Scene Description Network (denoted as MSDN), to solve the three vision tasks jointly in an end-to-end manner. Object, phrase, and caption regions are first aligned with a dynamic graph based on their spatial and semantic connections. Then a feature refining structure is used to pass messages across the three levels of semantic tasks through the graph. We benchmark the learned model on three tasks, and show the joint learning across three tasks with our proposed method can bring mutual improvements over previous models. Particularly, on the scene graph generation task, our proposed method outperforms the stateof- art method with more than 3% margin. Code has been made publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Algorithm AS 136: A K-Means Clustering Algorithm
Hartigan, J. A., & Wong, M. A. (1979) · Applied Statistics · doi:10.2307/2346830 · Cited by 14,523
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Cited by
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2018) · arXiv · doi:10.48550/arxiv.1809.02165 · Cited by 2,965
Self-citationDeep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965
Self-citationNeural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00611 · Cited by 1,052
Exploring Visual Relationship for Image Captioning
Yao, T., Pan, Y., Li, Y., & Mei, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_42 · Cited by 870
Self-citationGraph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_41 · Cited by 851
Unbiased Scene Graph Generation From Biased Training
Tang, K., Niu, Y., Huang, J., Shi, J., & Zhang, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00377 · Cited by 755
Learning to Compose Dynamic Tree Structures for Visual Contexts
Tang, K., Zhang, H., Wu, B., Luo, W., & Liu, W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00678 · Cited by 543
SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction
Zhang, P., Ouyang, W., Zhang, P., Xue, J., & Zheng, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01236 · Cited by 540
Self-citationTowards Diverse and Natural Image Descriptions via a Conditional GAN
Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.323 · Cited by 477
Knowledge-Embedded Routing Network for Scene Graph Generation
Chen, T., Yu, W., Chen, R., & Lin, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00632 · Cited by 432
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera
Armeni, I., He, Z.-Y., Zamir, A., Gwak, J., Malik, J., Fischer, M., & Savarese, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00576 · Cited by 350
Person Re-identification with Deep Similarity-Guided Graph Neural Network
Shen, Y., Li, H., Yi, S., Chen, D., & Wang, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_30 · Cited by 324
Self-citationA Comprehensive Survey of Scene Graphs: Generation and Application
Chang, X., Ren, P., Xu, P., Li, Z., Chen, X., & Hauptmann, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3137605 · Cited by 324
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
Chen, Y., Tai, L., Sun, K., & Li, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01211 · Cited by 316
Scene Graph Generation With External Knowledge and Image Reconstruction
Gu, J., Zhao, H., Lin, Z., Li, S., Cai, J., & Ling, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00207 · Cited by 308
Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs
Ji, J., Krishna, R., Fei-Fei, L., & Niebles, J. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01025 · Cited by 303
Learning Actor Relation Graphs for Group Activity Recognition
Wu, J., Wang, L., Wang, L., Guo, J., & Wu, G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01020 · Cited by 301
GPS-Net: Graph Property Sensing Network for Scene Graph Generation
Lin, X., Ding, C., Zeng, J., & Tao, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00380 · Cited by 287
Factorizable Net: An Efficient Subgraph-Based Framework for Scene Graph Generation
Li, Y., Ouyang, W., Zhou, B., Shi, J., Zhang, C., & Wang, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_21 · Cited by 283
Self-citationKimera: From SLAM to spatial perception with 3D dynamic scene graphs
Rosinol, A., Violette, A., Abate, M., Hughes, N., Chang, Y., Shi, J., Gupta, A., & Carlone, L. (2021) · The International Journal of Robotics Research · doi:10.1177/02783649211056674 · Cited by 271
Rahate, A., Walambe, R., Ramanna, S., & Kotecha, K. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.003 · Cited by 214
Learning to Generate Scene Graph from Natural Language Supervision
Zhong, Y., Shi, J., Yang, J., Xu, C., & Li, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00184 · Cited by 73
Self-citationVideo Object Grounding Using Semantic Roles in Language Description
Sadhu, A., Chen, K., & Nevatia, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01043 · Cited by 36
Visual Semantic Information Pursuit: A Survey
Liu, D., Bober, M., & Kittler, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2950025 · Cited by 39
Visual Question Generation: The State of the Art
Patil, C., & Patwardhan, M. (2021) · ACM Computing Surveys · doi:10.1145/3383465 · Cited by 25
Scene Graph Contrastive Learning for Embodied Navigation
Singh, K. P., Salvador, J., Weihs, L., & Kembhavi, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00999 · Cited by 23
Metadata
- Authors
- Li, Y., Ouyang, W., Zhou, B., Wang, K., & Wang, X.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1270-1279
- OpenAlex
- W2963649796
- Cited by
- 500
- References
- 53
- FWCI
- 15.29
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.