Toward Long Video Understanding via Fine-Detailed Video Story Generation
Abstract
Long video understanding has become a critical task in computer vision, driving advancements across numerous applications from surveillance to content retrieval. Existing video understanding methods suffer from two challenges when dealing with long video understanding: intricate long-context relationship modeling and interference from redundancy. To tackle these challenges, we introduce Fine-Detailed Video Story generation (FDVS), which interprets long videos into detailed textual representations. Specifically, to achieve fine-grained modeling of long-temporal content, we propose a Bottom-up Video Interpretation Mechanism that progressively interprets video content from clips to video. To avoid interference from redundant information in videos, we introduce a Semantic Redundancy Reduction mechanism that removes redundancy at both the visual and textual levels. Our method transforms long videos into hierarchical textual representations that contain multi-granularity information of the video. With these representations, FDVS is applicable to various tasks without any fine-tuning. We evaluate the proposed method across eight datasets spanning three tasks. The performance demonstrates the effectiveness and versatility of our method.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FDVS and LangRepo segment long videos into clips and generate textual summaries by integrating multi-source signals, which are then hierarchically aggregated into a global long video story.
“In the video memory domain, FDVS (You et al., 2024) and LangRepo (Kahatapitiya et al., 2025) segment long videos into clips and generate textual summaries by integrating multi-source signals such as subtitles, object detection, and scene descriptions, which are then hierarchically aggregated into a global long video story.”
From Memory in the Age of AI Agents (2025) · cites this paper as You et al., 2024
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · see citing works
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,810 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
mPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318 · see citing works
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,279 · see citing works
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., & Shum, H.-Y. (2022) · arXiv · Cited by 1,154 · cited 3× · see citing works
“Among these foundation models, GroundingDINO [71] and BLIP2 [65] are open-set models and are applicable to a wide range of categories.”
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,098 · see citing works
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., & Tang, J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.26 · Cited by 1,028 · see citing works
CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning
Luo, H., Ji, L., Zhong, M., Chen, Y., Lei, W., Duan, N., & Li, T. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.07.028 · Cited by 947 · see citing works
Self-citationVideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909 · cited 1× · see citing works
“The success of this approach relies on end-to-end training, utilizing pretext pretraining tasks such as masked language modeling [49], masked video modeling [50], [51], video-language masked modeling [52], video-text matching [53], and video-text contrastive learning [6], [54].”
TALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 884 · see citing works
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 832 · see citing works
Efficiently Modeling Long Sequences with Structured State Spaces
Gu, A., Goel, K., & Ré, C. (2021) · arXiv · doi:10.48550/arxiv.2111.00396 · Cited by 805 · see citing works
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · see citing works
Self-citationEnd-to-End Learning of Visual Representations From Uncurated Instructional Videos
Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Šivic, J., & Zisserman, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00990 · Cited by 622 · see citing works
Multi-modal Transformer for Video Retrieval
Gabeur, V., Sun, C., Alahari, K., & Schmid, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_13 · Cited by 615 · see citing works
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Zhang, H., Li, X. S., & Bing, L. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.49 · Cited by 601 · see citing works
Self-citationVideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., & Qiao, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01398 · Cited by 536 · see citing works
Self-citationVideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
Xu, H., Ghosh, G., Huang, P.-Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettlemoyer, L., & Feichtenhofer, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.544 · Cited by 468 · see citing works
Pareek, P., & Thakkar, A. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09904-8 · Cited by 466 · see citing works
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Maaz, M., Rasheed, H., Khan, S. H., & Khan, F. S. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.679 · Cited by 444 · see citing works
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
Lin, B., Ye, Y., Zhu, B., Cui, J., Ning, M., Jin, P., & Li, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.342 · Cited by 439 · see citing works
Video Question Answering via Gradually Refined Attention over Appearance and Motion
Xu, D., Zhao, Z., Xiao, J., Wu, F., Zhang, H., He, X., & Zhuang, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123427 · Cited by 438 · see citing works
A Joint Sequence Fusion Model for Video Question Answering and Retrieval
Yu, Y., Kim, J.-S., & Kim, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_29 · Cited by 347 · see citing works
SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
Lin, K., Li, L., Lin, C.-C., Ahmed, F., Gan, Z., Liu, Z., Lu, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01742 · Cited by 321 · see citing works
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions
Xiao, J., Shang, X., Yao, A., & Chua, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00965 · Cited by 315 · see citing works
Use What You Have: Video Retrieval Using Representations From Collaborative ExpertsPreprint
Liu, Y., Albanie, S., Nagrani, A., & Zisserman, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13487 · Cited by 310 · see citing works
Dual Encoding for Zero-Example Video Retrieval
Dong, J., Li, X., Xu, C., Ji, S., He, Y., Yang, G., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00957 · Cited by 292 · see citing works
Self-citationActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
Yu, Z., Xu, D., Yu, J., Yu, T., Zhao, Z., Zhuang, Y., & Tao, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019127 · Cited by 281 · cited 2× · see citing works
“ActivityNet-QA [61] is derived from ActivityNet dataset [64].”
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_27 · Cited by 275 · see citing works
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
Yang, A., Nagrani, A., Seo, P. H., Miech, A., Pont-Tuset, J., Laptev, I., Šivic, J., & Schmid, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01032 · Cited by 266 · see citing works
Scaling Up Vision-Language Pretraining for Image Captioning
Hu, X., Gan, Z., Wang, J., Yang, Z., Liu, Z., Lu, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01745 · Cited by 241 · see citing works
AdaFrame: Adaptive Frame Selection for Fast Video Recognition
Wu, Z., Xiong, C., Ma, C., Socher, R., & Davis, L. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00137 · Cited by 225 · see citing works
MERLOT: Multimodal Neural Script Knowledge ModelsPreprint
Zellers, R., Lu, X., Hessel, J., Yu, Y., Park, J. S., Cao, J., Farhadi, A., & Choi, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02636 · Cited by 224 · see citing works
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00171 · Cited by 220 · see citing works
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
Li, D., Li, J., Li, H., Niebles, J. C., & Hoi, S. C. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00490 · Cited by 178 · see citing works
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01725 · Cited by 176 · see citing works
Cross-Modal and Hierarchical Modeling of Video and Text
Zhang, B., Hu, H., & Sha, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_23 · Cited by 172 · see citing works
HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval
Liu, S., Fan, H., Qian, S., Chen, Y., Ding, W., & Wang, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01170 · Cited by 171 · see citing works
Self-citationAoE: Angle-optimized Embeddings for Semantic Textual Similarity
Li, X., & Li, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.101 · Cited by 169 · see citing works
Self-citationAll in One: Exploring Unified Video-Language Pre-Training
Wang, J., Ge, Y., Yan, R., Ge, Y., Lin, K. Q., Tsutsui, S., Lin, X., Cai, G., Wu, J., Shan, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00638 · Cited by 164 · see citing works
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01589 · Cited by 163 · see citing works
Question Generation via Overgenerating Transformations and Ranking
Heilman, M., & Smith, N. A. (2009) · Defense Technical Information Center (DTIC) · doi:10.21236/ada531042 · Cited by 150 · see citing works
Revisiting the “Video” in Video-Language Understanding
Buch, S., Eyzaguirre, C., Gaidon, A., Wu, J., Fei-Fei, L., & Niebles, J. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00293 · Cited by 139 · see citing works
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token ModelingPreprint
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12681 · Cited by 130 · see citing works
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and GenerationPreprint
Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Li, X., Guo, C., Chen, X., Wang, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.06942 · Cited by 126 · see citing works
Self-citationSelective Structured State-Spaces for Long-Form Video Understanding
Wang, J., Zhu, W., Wang, P., Yu, X., Liu, L. L., Omar, M., & Hamid, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00618 · Cited by 121 · see citing works
InternVideo: General Video Foundation Models via Generative and Discriminative LearningPreprint
Wang, Y., Li, K., Li, Y., He, Y., Huang, B., Zhao, Z., Zhang, H., Xu, J., Liu, Y., Wang, Z., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.03191 · Cited by 111 · see citing works
Self-citationVideo Corpus Moment Retrieval with Contrastive Learning
Zhang, H., Sun, A., Jing, W., Nan, G., Zhen, L., Zhou, J. T., & Goh, R. S. M. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462874 · Cited by 107 · see citing works
Video Graph Transformer for Video Question Answering
Xiao, J., Zhou, P., Chua, T., & Yan, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_3 · Cited by 99 · see citing works
VideoAgent: Long-Form Video Understanding with Large Language Model as Agent
Wang, X., Zhang, Y., Zohar, O., & Yeung, S. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_4 · Cited by 97 · see citing works
Zeng, R., Gan, C., Chen, P., Huang, W., Wu, Q., & Tan, M. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2922108 · Cited by 96 · see citing works
Self-citationMIST : Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering
Gao, D., Zhou, L., Ji, L., Zhu, L., Yang, Y., & Shou, M. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01419 · Cited by 96 · see citing works
Long Movie Clip Classification with State-Space Video Models
Islam, M. M., & Bertasius, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_6 · Cited by 95 · see citing works
MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions
Soldan, M., Pardo, A., Alcázar, J. L., Heilbron, F. C., Zhao, C., Giancola, S., & Ghanem, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00497 · Cited by 94 · see citing works
Revealing Single Frame Bias for Video-and-Language Learning
Lei, J., Berg, T. L., & Bansal, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.29 · Cited by 90 · see citing works
Graph Convolutional Module for Temporal Action Localization in Videos
Zeng, R., Huang, W., Tan, M., Rong, Y., Zhao, P., Huang, J., & Gan, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3090167 · Cited by 89 · see citing works
Self-citationZero-Shot Video Question Answering via Frozen Bidirectional Language ModelsPreprint
Yang, A., Miech, A., Šivic, J., Ivan Laptev, A., & Schmid, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.08155 · Cited by 87 · see citing works
Reading-Strategy Inspired Visual Representation Learning for Text-to-Video Retrieval
Dong, J., Wang, Y., Chen, X., Qu, X., Li, X., He, Y., & Wang, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3150959 · Cited by 84 · see citing works
Self-citationA Simple LLM Framework for Long-Range Video Question-Answering
Zhang, C., Lu, T., Islam, M. M., Wang, Z., Yu, S., Bansal, M., & Bertasius, G. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.1209 · Cited by 83 · see citing works
MGSampler: An Explainable Sampling Strategy for Video Action Recognition
Zhi, Y., Tong, Z., Wang, L., & Wu, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00154 · Cited by 80 · see citing works
EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding
Mangalam, K., Akshulakov, R., & Malik, J. (2023) · neural information processing systems · doi:10.52202/075280-2004 · Cited by 77 · see citing works
Language Models with Image Descriptors Are Strong Few-Shot Video-Language Learners
Wang, Z., Li, M., Xu, R., Zhou, L., Lei, J., Lin, X., Wang, S., Yang, Z., Zhu, C., Hoiem, D., et al. (2022) · neural information processing systems · doi:10.52202/068431-0617 · Cited by 76 · see citing works
Self-Chained Image-Language Model for Video Localization and Question Answering
Yu, S., Cho, J., Yadav, P., & Bansal, M. (2023) · neural information processing systems · doi:10.52202/075280-3354 · Cited by 74 · see citing works
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
Li, L., Gan, Z., Lin, K., Lin, C.-C., Liu, Z., Liu, C., & Wang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02214 · Cited by 73 · see citing works
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training
Ye, Q., Xu, G., Yan, M., Xu, H., Qian, Q., Zhang, J., & Huang, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01413 · Cited by 70 · see citing works
Partially Relevant Video Retrieval
Dong, J., Chen, X., Zhang, M., Yang, X., Chen, S., Li, X., & Wang, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547976 · Cited by 69 · see citing works
Self-citationUnsupervised Pre-training for Temporal Action Localization Tasks
Zhang, C., Yang, T., Weng, J., Cao, M., Wang, J., & Zou, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01364 · Cited by 62 · see citing works
Verbs in Action: Improving verb understanding in video-language models
Momeni, L., Caron, M., Nagrani, A., Zisserman, A., & Schmid, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01428 · Cited by 58 · see citing works
VideoGraph: Recognizing Minutes-Long Human Activities in VideosPreprint
Hussein, N., Gavves, E., & Smeulders, A. W. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05143 · Cited by 56 · see citing works
Dynamic Spatial Focus for Efficient Compressed Video Action Recognition
Zheng, Z., Yang, L., Wang, Y., Zhang, M., He, L., Huang, G. M., & Li, F. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3287201 · Cited by 54 · see citing works
Self-citationLi, H., Liu, M., Hu, Z., Nie, F., & Yu, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3246091 · Cited by 54 · see citing works
Video Moment Retrieval via Comprehensive Relation-Aware Network
Xin, S., Gao, J., Zhu, Y., Wang, X., & Zhou, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3250518 · Cited by 51 · see citing works
LocVTP: Video-Text Pre-training for Temporal Localization
Cao, M., Yang, T., Weng, J., Zhang, C., Wang, J., & Zou, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_3 · Cited by 47 · see citing works
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
Sun, Y., Xue, H., Song, R., Liu, B., Yang, H., & Fu, J. (2022) · neural information processing systems · doi:10.52202/068431-2756 · Cited by 45 · see citing works
EclipSE: Efficient Long-Range Video Retrieval Using Sight and Sound
Lin, Y.-B., Lei, J., Bansal, M., & Bertasius, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_24 · Cited by 39 · see citing works
Temporal Distinct Representation Learning for Action Recognition
Weng, J., Luo, D., Wang, Y., Tai, Y. Y., Wang, C., Li, J., Huang, F., Jiang, X., & Yuan, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_22 · Cited by 35 · see citing works
Self-citationJoint Searching and Grounding: Multi-Granularity Video Content Retrieval
Chen, Z., Jiang, X., Xu, X., Cao, Z., Mo, Y., & Shen, H. T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612349 · Cited by 32 · see citing works
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
Dong, J., Zhang, M., Zhang, Z., Chen, X., Liu, D., Qu, X., Wang, X., & Liu, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01038 · Cited by 30 · see citing works
Cross Time-Frequency Transformer for Temporal Action Localization
Yang, J., Wei, P., & Zheng, N. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3326692 · Cited by 29 · see citing works
Video Question Answering With Semantic Disentanglement and Reasoning
Liu, J., Wang, G., Xie, J., Zhou, F., & Xu, H. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3317447 · Cited by 28 · see citing works
Sun, W., Su, R., Yu, Q., & Xu, D. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3201540 · Cited by 27 · see citing works
Complementarity-Aware Space Learning for Video-Text Retrieval
Zhu, J., Zeng, P., Gao, L., Li, G., Liao, D., & Song, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3235523 · Cited by 25 · see citing works
Multi-Granularity Interaction and Integration Network for Video Question Answering
Wang, Y., Liu, M., Wu, J., & Nie, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3278492 · Cited by 23 · see citing works
Self-citationCompressed Video Action Recognition With Dual-Stream and Dual-Modal Transformer
Mou, Y., Jiang, X., Xu, K., Sun, T., & Wang, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3319140 · Cited by 21 · see citing works
Multilevel Semantic Interaction Alignment for Video–Text Cross-Modal Retrieval
Chen, L., Deng, Z., Liu, L., & Yin, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3360530 · Cited by 21 · see citing works
Localizing Moments in Long Video Via Multimodal Guidance
Barrios, W., Soldan, M., Ceballos-Arroyo, A. M., Heilbron, F. C., & Ghanem, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01257 · Cited by 18 · see citing works
Self-Supervised Adversarial Video Summarizer With Context Latent Sequence Learning
Xu, Y., Li, X., Pan, L., Sang, W., Wei, P., & Zhu, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3240464 · Cited by 17 · see citing works
Self-citationYang, Z., An, G., Zheng, Z., Cao, S., & Ruan, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3390133 · Cited by 16 · see citing works
EvCap: Element-Aware Video Captioning
Liu, S., Li, A., Zhao, Y., Wang, J., & Wang, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3399933 · Cited by 16 · see citing works
Self-citationAlignment-Guided Temporal Attention for Video Action Recognition
Zhao, Y., Li, Z., Guo, X., & Lu, Y. (2022) · neural information processing systems · doi:10.52202/068431-0990 · Cited by 14 · see citing works
Supervoxel Attention Graphs for Long-Range Video Modeling
Wang, Y., Bertasius, G., Oh, T.-H., Gupta, A. K., Hoai, M., & Torresani, L. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00020 · Cited by 10 · see citing works
Self-citationMemory Consolidation Enables Long-Context Video UnderstandingPreprint
Balažević, I., Shi, Y., Papalampidi, P., Chaabouni, R., Koppula, S., & Hénaff, O. J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.05861 · Cited by 9 · see citing works
Truncate-Split-Contrast: A Framework for Learning from Mislabeled Videos
Wang, Z., Weng, J., Yuan, C., & Wang, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25375 · Cited by 4 · see citing works
Flagged first, then by citations
3 of the 103 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoning
Wang, Z., Chen, B., Yue, Z., Wang, Y., Qiao, Y., Wang, L., & Wang, L. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i13.38018 · Cited by 5 · see citing works
Self-citationTian, Q., Zeng, F., Ning, J., & Zhang, L. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104456 · Cited by 4 · see citing works
Zhang, C., Xu, S., & Xiao, X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104457 · Cited by 3 · see citing works
Spatial–Temporal Clue Reasoning Chain for Long Video Question Answering
Gong, H., Yan, C., Zhang, J., Sun, Y., Gao, Y., & Li, L. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3609441 · Cited by 3 · see citing works
Multi-Teacher Temporal Regulation Network for Surgical Workflow Recognition
Huang, K., Yuan, X., Liu, R.-D., Ye, L., Zhou, Y., Hu, B., & Zhang, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3580082 · Cited by 2 · see citing works
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 1 · see citing works
Self-citationHou, J., Liu, C., Liu, D., Hull, V., Wang, Y., Zhao, X., Tan, Y., Shi, X., Cheng, Y., Tang, Z., et al. (2026) · Animals · doi:10.3390/ani16060943 · Cited by 1 · see citing works
Self-citationYe, G., Zhao, H., Shen, Y., Li, J., Li, F., Jiang, Z., & Li, K. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3676204 · Cited by 1 · see citing works
SegCL: Segmented Reasoning with Global Visual-Audio Knowledge for Complex Long Video Understanding
Yang, S., Lü, N., Wang, B., Li, Y., & Cui, R. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5567-3_3
Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data
Gabeff, V., Maquignaz, B., Shan, J., Mamooler, S., Sümbül, G., Costelloe, B. R., Tuia, D., & Mathis, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37574-2_3
A Survey of Multi-Model Collaboration in Video Understanding
Chen, Y., Zhang, J., Zhang, L., Liu, C., Gao, R., Lu, Z., Qi, J., & Lei, Q. (2026) · Data · doi:10.3390/data11090230
Self-citationCaptAin: Caption-Driven Alignment for Bridging Modality Gaps in Partially Relevant Video Retrieval
Chen, C., Zhou, K., Wang, F., Ning, Y., Xiong, Z., Li, Y., Wen, Z., & Tan, M. (2026) · doi:10.1109/cvprf71909.2026.00596
Self-citationHan, P. (2026) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2026.1934032
- Authors
- You, Z., Wen, Z., Chen, Y., Li, X., Zeng, R., Wang, Y., & Tan, M.
- Affiliations
- South China University of Technology (CN)
- Peng Cheng Laboratory (CN)
- Shenzhen MSU-BIT University (CN)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2025-05
- Volume
- 35
- Issue
- 5
- Pages
- 4592-4607
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W4405232240
- Cited by
- 11
- References
- 103
- Citations by year
- 2026: 7 · 2025: 4
- FWCI
- 1.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- Funders
- National Natural Science Foundation of China (62072190, 62202311)
- Key-Area Research and Development Program Guangdong Province (2018B010107001)
- Major Key Project of Peng Cheng Laboratory (PCL2023A08)
- TCL Science and Technology Innovation Fund, Shenzhen Natural Science Foundation (20220809180405001)
- Excellent Science and Technology Creative Talent Training Program of Shenzhen Municipality (RCBS20221008093224017)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- long video understanding, video summarization, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
