Deep Reasoning with Knowledge Graph for Social Relationship Understanding
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Social relationships (e.g., friends, couple etc.) form the basis of the social network in our daily life. Automatically interpreting such relationships bears a great potential for the intelligent systems to understand human behavior in depth and to better interact with people at a social level. Human beings interpret the social relationships within a group not only based on the people alone, and the interplay between such social relationships and the contextual information around the people also plays a significant role. However, these additional cues are largely overlooked by the previous studies. We found that the interplay between these two factors can be effectively modeled by a novel structured knowledge graph with proper message propagation and attention. And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects. Meanwhile, a graph attentional mechanism is introduced to explicitly reason about the discriminative objects to promote recognition. Extensive experiments on the public benchmarks demonstrate the superiority of our method over the existing leading competitors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Similar to [ Li et al. , 2017 ] , we utilize the widely used ResNet-101 [ He et al. , 2016 ] and VGG-16 [ Simonyan and Zisserman, 2014 ] to extract features for person regions and semantic object regions respectively.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 2×
“…the other nodes and from the previous timestep to update each node’s hidden state through a gating mechanism similar to the Gated Recurrent Unit [Cho et al. , 2014; Li et al. , 2015b], formulated as where σ and tanh are the logistic sigmoid and hyperbolic tangent functions, respectively, and…”
MethodModeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483 · cited 2×
“In computer vision community, social information has been considered as supplementary cues to improve various tasks including multi-target tracking [Choi and Savarese, 2012], human trajectory prediction [Alahi et al. , 2016] and group activity analysis [Lan et al. , 2012; Deng et al. , 2016].”
BackgroundMethodConvolutional Networks on Graphs for Learning Molecular Fingerprints
Duvenaud, D., Maclaurin, D., Aguilera-Iparraguirre, J., Gómez-Bombarelli, R., Hirzel, T., Aspuru-Guzik, A., & Adams, R. P. (2015) · arXiv · doi:10.48550/arxiv.1509.09292 · Cited by 2,628
The four elementary forms of sociality: Framework for a unified theory of social relations.
Fiske, A. P. (1992) · Psychological Review · doi:10.1037/0033-295x.99.4.689 · Cited by 2,510 · cited 1×
“Concurrently, [Li et al. , 2017] follow relational models theory [Fiske, 1992] to define a hierarchical social relationship categories, which involve 3 coarse-level and 6 fine-level relationships.”
BackgroundVisual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,069 · cited 2×
“In image analysis tasks, most works are dedicated to recognizing visual attributes [ Huang et al. , 2016 ] and visual relationships [ Lu et al. , 2016 ] .”
Method
Cited by
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880
Graph Neural Networks: A Review of Methods and Applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2018) · arXiv · doi:10.48550/arxiv.1812.08434 · Cited by 6,880
A review: Knowledge reasoning over knowledge graph
Chen, X., Jia, S., & Xiang, Y. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112948 · Cited by 1,035
Knowledge Graphs: Opportunities and Challenges
Peng, C., Xia, F., Naseriparsa, M., & Osborne, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10465-9 · Cited by 717
Xia, F., Sun, K., Yu, S., Aziz, A., Wan, L., Pan, S., & Liu, H. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3076021 · Cited by 473
Knowledge-Embedded Routing Network for Scene Graph Generation
Chen, T., Yu, W., Chen, R., & Lin, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00632 · Cited by 432
Self-citationPyramid Graph Networks With Connection Attentions for Region-Based One-Shot Semantic Segmentation
Zhang, C., Lin, G., Liu, F., Guo, J., Wu, Q., & Yao, R. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00968 · Cited by 346
Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition
Chen, T., Xu, M., Hui, X., Wu, H., & Lin, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00061 · Cited by 332
Self-citationIntelligent financial fraud detection practices in post-pandemic era
Zhu, X., Ao, X., Qin, Z., Chang, Y., Liu, Y., He, Q., & Li, J. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100176 · Cited by 197
“Powerful knowledge reasoning technologies based on knowledge graphs can help mine the secret relationship between entities connected with fraud and provide potential evidence to make up for missing information.(155,156) Thus, the knowledge graph will be one of the most important and promising tools that mine valuable…”
Metadata
- Authors
- Wang, Z., Chen, T., Ren, J., Yu, W., Cheng, H., & Lin, L.
- Venue
- Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
- Published
- 2018-07
- Pages
- 1021-1028
- OpenAlex
- W2963175158
- Cited by
- 178
- References
- 33
- FWCI
- 8.22
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.