Target Adaptive Context Aggregation for Video Scene Graph Generation
Open accessAbstract
This paper deals with a challenging task of video scene graph generation (VidSGG), which could serve as a structured video representation for high-level understanding tasks. We present a new detect-to-track paradigm for this task by decoupling the context modeling for relation prediction from the complicated low-level entity tracking. Specifically, we design an efficient method for frame-level VidSGG, termed as Target Adaptive Context Aggregation Network (TRACE), with a focus on capturing spatio-temporal context information for relation recognition. Our TRACE framework streamlines the VidSGG pipeline with a modular design, and presents two unique blocks of Hierarchical Relation Tree (HRTree) construction and Target-adaptive Context Aggregation. More specific, our HRTree first provides an adpative structure for organizing possible relation candidates efficiently, and guides context aggregation module to effectively capture spatio-temporal structure information. Then, we obtain a contextualized feature representation for each relation candidate and build a classification head to recognize its relation category. Finally, we provide a simple temporal association strategy to track TRACE detected results to yield the video-level VidSGG. We perform experiments on two VidSGG benchmarks: ImageNet-VidVRD and Action Genome, and the results demonstrate that our TRACE achieves the state-of-the-art performance. The code and models are made available at https://github.com/MCG-NJU/TRACE.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Aligning a sequence of instructions to a video of someone carrying out a task is an example of a powerful pretraining pretext task.
“Aligning a sequence of instructions to a video of someone carrying out a task is an example of a powerful pretraining pretext task [7], [156].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [156]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
The Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 3,711 · see citing works
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.515 · Cited by 1,507 · see citing works
Scene Graph Generation by Iterative Message Passing
Xu, D., Zhu, Y., Choy, C. B., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.330 · Cited by 1,271 · see citing works
Visual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M. S., & Fei-Fei, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,118 · see citing works
Image retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,098 · see citing works
Neural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00611 · Cited by 1,095 · see citing works
Temporal Segment Networks for Action Recognition in Videos
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868668 · Cited by 928 · see citing works
Self-citationGraph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_41 · Cited by 889 · see citing works
TALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 884 · see citing works
BMN: Boundary-Matching Network for Temporal Action Proposal Generation
Lin, T., Liu, X., Li, X., Ding, E., & Wen, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00399 · Cited by 694 · see citing works
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646 · see citing works
Learning to Compose Dynamic Tree Structures for Visual Contexts
Tang, K., Zhang, H., Wu, B., Luo, W., & Liu, W. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00678 · Cited by 558 · see citing works
TVQA: Localized, Compositional Video Question Answering
Lei, J., Yu, L., Bansal, M., & Berg, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1167 · Cited by 541 · cited 1× · see citing works
“This structured video representation, known as video scene graph [27], can contribute to more accurate action recognition [10] and allow for our vision models to tackle high-level and complex inference tasks, such as video caption [7, 36], video retrieval [4, 16], and video question answering [33, 15].”
BackgroundGuadarrama, S., Krishnamoorthy, N., Malkarnenkar, G., Venugopalan, S., Mooney, R. J., Darrell, T., & Saenko, K. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.337 · Cited by 500 · see citing works
Appearance-and-Relation Networks for Video Classification
Wang, L., Li, W., Li, W., & Gool, L. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00155 · Cited by 407 · see citing works
Self-citationAction Genome: Actions As Compositions of Spatio-Temporal Scene Graphs
Ji, J., Krishna, R., Fei-Fei, L., & Niebles, J. C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01025 · Cited by 326 · see citing works
Factorizable Net: An Efficient Subgraph-Based Framework for Scene Graph Generation
Li, Y., Ouyang, W., Zhou, B., Shi, J., Zhang, C., & Wang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_21 · Cited by 290 · see citing works
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_27 · Cited by 275 · see citing works
Graphical Contrastive Losses for Scene Graph Parsing
Zhang, J., Shih, K. J., Elgammal, A., Tao, A., & Catanzaro, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01180 · Cited by 257 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity UnderstandingPreprint
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01753 · Cited by 221 · see citing works
Relaxed Transformer Decoders for Direct Action Proposal Generation
Tan, J., Tang, J., Wang, L., & Wu, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01327 · Cited by 203 · see citing works
Self-citationZoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
Yin, G., Sheng, L., Liu, B., Yu, N., Wang, X., Shao, J., & Loy, C. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_20 · Cited by 155 · see citing works
Video Visual Relation Detection
Shang, X., Ren, T., Guo, J., Zhang, H., & Chua, T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123380 · Cited by 147 · see citing works
Li, Y., Wang, Z., Wang, L., & Wu, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_5 · Cited by 126 · see citing works
Self-citationVideo Relationship Reasoning Using Gated Spatio-Temporal Energy Graph
Tsai, Y.-H. H., Divvala, S., Morency, L., Salakhutdinov, R., & Farhadi, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01067 · Cited by 98 · see citing works
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01211-2 · Cited by 90 · see citing works
Self-citationVideo Relation Detection with Spatio-Temporal Graph
Qian, X., Zhuang, Y., Li, Y., Xiao, S., Pu, S., & Xiao, J. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351058 · Cited by 82 · see citing works
Beyond Short-Term Snippet: Video Relation Detection With Spatio-Temporal Global Context
Liu, C., Yang, J., Xu, K., Gong, G., & Mu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01085 · Cited by 75 · see citing works
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation
Wang, W., Wang, R., Shan, S., & Chen, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_14 · Cited by 47 · see citing works
Video Relation Detection via Multiple Hypothesis Association
Su, Z., Shang, X., Chen, J., Jiang, Y., Qiu, Z., & Chua, T. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413764 · Cited by 37 · see citing works
Modelling Sentence Pairs with Tree-structured Attentive EncoderPreprint
Zhou, Y., Liu, C., & Pan, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02806 · Cited by 31 · see citing works
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph GenerationPreprint
Wang, W., Wang, R., Shan, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08760 · Cited by 10 · see citing works
Multiple Hypothesis Video Relation Detection
Di, D., Shang, X., Zhang, W., Yang, X., & Chua, T. (2019) · IEEE International Conference on Multimedia Big Data · doi:10.1109/bigmm.2019.000-9 · Cited by 9 · see citing works
Relaxed Transformer Decoders for Direct Action Proposal GenerationPreprint
Tan, J., Tang, J., Wang, L., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01894 · Cited by 4 · see citing works
Self-citationChen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
RelTR: Relation Transformer for Scene Graph Generation
Cong, Y., Yang, M. Y., & Rosenhahn, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268066 · Cited by 251 · see citing works
Scene Graph Generation: A comprehensive survey
Li, H., Zhu, G., Zhang, L., Jiang, Y., Dang, Y., Hou, H., Shen, P., Zhao, X., Shah, S. A. A., & Bennamoun, M. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127052 · Cited by 114 · see citing works
Structured Sparse R-CNN for Direct Scene Graph Generation
Teng, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01883 · Cited by 78 · see citing works
Self-citationDynamic Scene Graph Generation via Anticipatory Pre-training
Li, Y., Yang, X., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01350 · Cited by 45 · see citing works
Panoptic Video Scene Graph Generation
Yang, J., Peng, W., Li, X., Guo, Z., Chen, L., Li, B., Ma, Z., Zhou, K., Zhang, W., Loy, C. C., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01791 · Cited by 45 · see citing works
Unbiased Scene Graph Generation in Videos
Nag, S., Min, K., Tripathi, S., & Roy–Chowdhury, A. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02184 · Cited by 45 · see citing works
Classification-Then-Grounding: Reformulating Video Scene Graphs as Temporal Bipartite Graphs
Gao, K., Chen, L., Niu, Y., Shao, J., & Xiao, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01889 · Cited by 40 · see citing works
Dynamic Scene Graph Generation via Temporal Prior Inference
Wang, S., Gao, L., Lyu, X., Guo, Y., Zeng, P., & Song, J. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548324 · Cited by 31 · see citing works
Spatial–Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation
Pu, T., Chen, T., Wu, H., Lu, Y., & Lin, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3345652 · Cited by 30 · see citing works
Exploiting Long-Term Dependencies for Generating Dynamic Scene Graphs
Feng, S., Mostafa, H., Nassar, M., Majumdar, S., & Tripathi, S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00510 · Cited by 30 · see citing works
Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation
Wang, J., Wen, Z., Li, X., Guo, Z., Yang, J., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3442301 · Cited by 28 · see citing works
Explore Contextual Information for 3D Scene Graph Generation
Liu, Y., Long, C., Zhang, Z., Liu, B., Zhang, Q., Yin, B., & Yang, X. S. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3219451 · Cited by 27 · see citing works
Interaction Region Visual Transformer for Egocentric Action Anticipation
Roy, D., Rajendiran, R., & Fernando, B. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00660 · Cited by 25 · see citing works
Meta Spatio-Temporal Debiasing for Video Scene Graph Generation
Li, X., Qu, H., Kuen, J., Gu, J., & Liu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_22 · Cited by 22 · see citing works
OED: Towards One-stage End-to-End Dynamic Scene Graph Generation
Wang, G., Li, Z., Chen, Q., & Liu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02639 · Cited by 18 · see citing works
Self-citationSceneLLM: Implicit language reasoning in LLM for dynamic scene graph generation
Zhang, H., Li, Z., & Liu, J. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111992 · Cited by 15 · see citing works
Self-citationEnd-to-End Video Scene Graph Generation With Temporal Propagation Transformer
Zhang, Y., Pan, Y., Yao, T., Huang, R., Mei, T., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3283879 · Cited by 15 · see citing works
Cross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs
Wang, J., Huang, J., Zhang, C., & Deng, Z. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161478 · Cited by 14 · see citing works
LABRAD-OR: Lightweight Memory Scene Graphs for Accurate Bimodal Reasoning in Dynamic Operating Rooms
Özsoy, E., Czempiel, T., Holm, F., Pellegrini, C., & Navab, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43996-4_29 · Cited by 13 · see citing works
TD²-Net: Toward Denoising and Debiasing for Video Scene Graph Generation
Lin, X., Shi, C., Zhan, Y., Yang, Z., Wu, Y., & Tao, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28137 · Cited by 12 · see citing works
SportsHHI: A Dataset for Human-Human Interaction Detection in Sports Videos
Wu, T., He, R., Wu, G., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01754 · Cited by 12 · see citing works
Self-citationQiu, Y., Nagasaki, Y., Hara, K., Kataoka, H., Suzuki, R., Iwata, K., & Satoh, Y. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00335 · Cited by 11 · see citing works
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation
Nguyen, T.-T., Nguyen, P., Cothren, J. D., Yılmaz, A., & Luu, K. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02714 · Cited by 11 · see citing works
A Survey on 3D Scene Graphs: Definition, Generation and Application
Bae, J., Shin, D., Ko, K., Lee, J., & Kim, U.-H. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-26889-2_13 · Cited by 10 · see citing works
When I Fall in Love: Capturing Video-Oriented Social Relationship Evolution via Attentive GNN
Qin, P., Wu, S., Xu, T., Hao, Y., Feng, F., Zhu, C., & Chen, E. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3337838 · Cited by 9 · see citing works
CAGNet: a context-aware graph neural network for detecting social relationships in videos
Yu, F., Fang, Y., Zhao, Z., Bei, J., Ren, T., & Wu, G. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00056-9 · Cited by 9 · see citing works
Self-citationOpen-Vocabulary Video Scene Graph Generation via Union-aware Semantic Alignment
Wu, Z., Gao, J., & Xu, C. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681061 · Cited by 9 · see citing works
LANDMARK: language-guided representation enhancement framework for scene graph generation
Chang, X., Wang, T., Cai, S., & Sun, C. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04722-1 · Cited by 9 · see citing works
Wang, W., Luo, Y., Chen, Z., Jiang, T., Yang, Y., & Xiao, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3338078 · Cited by 9 · see citing works
Prior Knowledge-driven Dynamic Scene Graph Generation with Causal Inference
Lu, J., Chen, L., Song, Y., Lin, S., Wang, C., & He, G. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612249 · Cited by 8 · see citing works
Triple Correlations-Guided Label Supplementation for Unbiased Video Scene Graph Generation
Wang, W., Gao, K., Luo, Y., Jiang, T., Gao, F., Shao, J., Sun, J., & Xiao, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612024 · Cited by 8 · see citing works
DiffVsgg: Diffusion-Driven Online Video Scene Graph Generation
Chen, M., Li, L., Wang, W., & Yang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02715 · Cited by 7 · see citing works
Entity Dependency Learning Network With Relation Prediction for Video Visual Relation Detection
Zhang, G., Tang, Y., Zhang, C., Zheng, X., & Zhao, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3437437 · Cited by 6 · see citing works
FloCoDe: Unbiased Dynamic Scene Graph Generation with Temporal Consistency and Correlation Debiasing
Khandelwal, A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00258 · Cited by 6 · see citing works
Unbiased Video Scene Graph Generation via Visual and Semantic Dual Debiasing
Li, Y., Li, Z., Chen, H., & Xu, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01774 · Cited by 5 · see citing works
Self-citationHAtt-Flow: Hierarchical Attention-Flow Mechanism for Group-Activity Scene Graph Generation in Videos
Chappa, N. V. S. R., Nguyen, P., Le, N., Dobbs, P. D., & Luu, K. (2024) · Sensors · doi:10.3390/s24113372 · Cited by 5 · see citing works
Concept-Enhanced Relation Network for Video Visual Relation Inference
Cao, Q., Huang, H., Ren, M., & Yuan, C. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3220426 · Cited by 5 · see citing works
Weakly-supervised Video Scene Graph Generation via Unbiased Cross-modal Learning
Wu, Z., Gao, J., & Xu, C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612019 · Cited by 5 · see citing works
Composite Relationship Fields with Transformers for Scene Graph Generation
Adaimi, G., Mizrahi, D., & Alahi, A. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00014 · Cited by 4 · see citing works
Chaotic World: A Large and Challenging Benchmark for Human Behavior Understanding in Chaotic Events
Ong, K. E., Ng, X. L., Li, Y., Ai, W., Zhao, K., Yeo, S. Y., & Liu, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01849 · Cited by 4 · see citing works
Video Visual Relation Detection With Contextual Knowledge Embedding
Cao, Q., & Huang, H. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3270328 · Cited by 4 · see citing works
Xu, Z., Lei, T., Li, Z., Wang, G., Chen, Q., Peng, Y., & Liu, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01467 · Cited by 3 · see citing works
Self-citationSituational Scene Graph for Structured Human-Centric Situation Understanding
Sugandhika, C., Li, C., Rajan, D., & Fernando, B. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00893 · Cited by 3 · see citing works
From a Visual Scene to a Virtual Representation: A Cross-Domain Review
Pereira, A., Carvalho, P., Pereira, N., Viana, P., & Côrte-Real, L. (2023) · IEEE Access · doi:10.1109/access.2023.3283495 · Cited by 4 · see citing works
GTR: A Grafting-Then-Reassembling Framework for Dynamic Scene Graph Generation
Liang, J., Wang, Y., Wang, Z., Liu, M., Fu, R., Wang, Z., & Qin, B. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/131 · Cited by 2 · see citing works
Video Scene Graph Generation with Spatial-Temporal Knowledge
Pu, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3613433 · Cited by 2 · see citing works
RealGraph: A Multiview Dataset for 4D Real-world Context Graph Generation
Lin, H., Chen, Z., Zhang, J., Bai, B., Wang, Y., Huang, R., & Fang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00348 · Cited by 2 · see citing works
Hu, Y., Zhang, F., Wei, R., & Gao, J. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01767-9 · Cited by 1 · see citing works
Cross-modality-enhanced visual Scene Graph Generation
Yu, F., Ji, H., & Li, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103430 · Cited by 1 · see citing works
Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization
Park, J., Lee, J., & Sohn, K. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02577-2 · Cited by 1 · see citing works
Iterative Learning with Extra and Inner Knowledge for Long-tail Dynamic Scene Graph Generation
Li, Y., Yang, X., & Xu, C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612430 · Cited by 1 · see citing works
STDG: Semi-Teacher-Student Training Paradigm for Depth-guided One-stage Scene Graph Generation
Zhou, X., Song, Z., He, J., Liu, H., & Fan, Z. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658024 · Cited by 1 · see citing works
Spatial-Temporal Saliency Guided Unbiased Contrastive Learning for Video Scene Graph Generation
Zhuang, W., Dong, B., Zhu, Z., Li, Z., Liu, J., Wang, Y., Hong, X., Li, X., & Zuo, W. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557688 · Cited by 1 · see citing works
Self-citationDDS: Decoupled Dynamic Scene-Graph Generation Network
Iftekhar, A., Ruschel, R., Kumar, S., You, S. O., & Manjunath, B. S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00936 · Cited by 1 · see citing works
A Novel Two-Stage Training Method for Unbiased Scene Graph Generation via Distribution Alignment
Jia, D., Zhou, M., Wei, W., Wang, D., & Bai, Z. (2023) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2023.12.009
Amirat, A., Baha, N., Bentarzi, M., & Bessai, Z. (2024) · International Conference on Innovative and Intelligent Information Technologies (IC3IT) · doi:10.1109/ic3it63743.2024.10869430
Lightweight Adaptive Perception Attention Model
Wu, J., Yang, J., Lu, J., & Liu, J. (2024) · International Conference on Wavelet Analysis and Pattern Recognition · doi:10.1109/icwapr63074.2024.10870518
Scene Graph Generation Based on Depth Information and Feature Enhancement
Liao, J., Qing, L., Li, L., Wang, P., Chen, H., & Zhao, W. (2024) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc62609.2024.10941810
Wu, R., Xu, J., Zheng, H., Xu, Z., Li, Z., Cheng, S., & Zhang, S. (2025) · Digital Communications and Networks · doi:10.1016/j.dcan.2025.04.010
Self-citationMEDP: Multimodal-Enhanced Dynamic Prototype learning for few-shot dynamic scene graph generation
Wang, X., Huang, Z., Meng, W., Wang, C., & He, G. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116323
Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistance
Choi, D., Kwon, T., Yang, D., Kim, H., & Yeo, J. (2025) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct68609.2025.00028
Fourier–Transformer Mixer Network for Efficient Video Scene Graph Prediction
Qu, D., & Ma, Y. (2026) · IEEE International Conference on Knowledge Innovation and Invention (ICKII) · doi:10.3390/engproc2025120016
Temporal Prompting with Vision-Language Models for Consistent Video Scene Graph Generation
Lee, D., Jang, C., Lim, S.-K., & Lee, C.-E. (2025) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc66702.2025.11388404
Revisiting Weakly-Supervised Video Scene Graph Generation via Pair Affinity Learning
Kang, M., Lee, M., Kim, M., Lee, J., Kim, D., Woo, S., Jeon, I., & Lee, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37132-4_20
MoSA: Motion-Guided Semantic Alignment for Dynamic Scene Graph Generation
Wang, X., Zhang, B., Wang, C., & He, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461631
Frequency-Guided Multi-Level Reasoning for Scene Graph Generation in Video
Li, C., Duan, Y., & Tao, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464376
FlowSGG: A Single-Stage Framework for Dynamic Scene Graph Generation Via Temporal Propagation
Hu, Y., Zhou, X., & Wang, C. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462060
End-to-End Entity-Predicate Association Reasoning for Dynamic Scene Graph Generation
Wang, L., Zhang, Y., Lü, T., Liu, F., Zhang, H. D., X, J., & Zhou, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01647
Self-citationUnified Scene Graph Forecasting Considering Node Birth and Disappearance
Seino, T., Saito, N., Ogawa, T., Asamizu, S., & Haseyama, M. (2026) · IEEE International Conference on Consumer Technology (ICCT-Pacific) · doi:10.1109/icct-pacific69083.2026.11519043
Liu, L., Li, G., Xie, Q., Wang, Z., Kuang, L., Xia, Y., & Yin, J. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3701841
Nguyen, N., Tran, C., Tran, A., & Pham, C. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104883
Joutei, B. T., Riand, M., Callet, P. L., Bruckert, A., & Dollé, L. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630234
Synthetic Visual Genome: Extracting Large-Scale Spatio-Temporal Scene Graphs from Videos
Gao, Z., Zhang, J., Ikechukwu, W. O., Park, J. S., You, T., Ogbu, D., Zheng, C., Huang, W., Yang, Y., Han, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37393-9_2
- Authors
- Teng, Y., Wang, L., Li, Z., & Wu, G.
- Affiliations
- Nanjing University (CN)
- Tencent (China) (CN)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13668-13677
- OpenAlex
- W3193902142
- Cited by
- 74
- References
- 50
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 14 · 2023: 27 · 2022: 7
- FWCI
- 3.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China
- Tencent
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- relation prediction, context aggregation, spatio-temporal context, entity tracking, temporal association, traces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
