MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Open accessAbstract
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks. Yet, existing systems can only handle videos with very few frames. For long videos, the computation complexity, memory cost, and long-term temporal connection impose additional challenges. Taking advantage of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination with our specially designed memory mechanism, we propose the MovieChat to overcome these challenges. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1K benchmark with 1K long video and 14K manual annotations for validation of the effectiveness of our method. The code, models and data can be found in https://reself.github.io/MovieChat.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
By making plans and structured environment representations the readable and writable core of working memory, agents can maintain goal consistency and revise strategies robustly against perception failures.
“By making plans and structured environment representations the readable and writable core of working memory, agents can maintain goal consistency and revise strategies robustly against perception failures (Song et al., 2023).”
From Memory in the Age of AI Agents (2025) · cites this paper as Song et al., 2023
Sections 1 (Introduction) and 3.3 (Short-term Memory) describe buffers of visual tokens for long-video understanding, not plans or structured environment representations used as readable and writable working memory. A search for goal consistency, strategy revision, and robustness to perception failures found no passage addressing those claims. Page numbers were not available in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · see citing works
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A., Zhao, J.-Q., Wang, W., Li, B. A., Fung, P., & Hoi, S. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.06500 · Cited by 4,191 · see citing works
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
Self-citationLiu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294 · see citing works
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,810 · see citing works
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
mPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318 · see citing works
Birch, A., & Osborne, M. (2011) · The Association for Computational Linguistics · Cited by 1,196 · see citing works
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986 · see citing works
Collecting Highly Parallel Data for Paraphrase Evaluation
Chen, D. J., & Dolan, W. B. (2011) · Cited by 898 · see citing works
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863 · see citing works
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction ModelPreprint
Gao, P., Han, J., Zhang, R., Lin, Z.-Y., Geng, S., Zhou, A., Zhang, W., Lu, P., He, C.-H., Yue, X., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.15010 · Cited by 777 · see citing works
Squire, L. R., Genzel, L., Wixted, J. T., & Morris, R. G. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021766 · Cited by 700 · see citing works
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646 · see citing works
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Zhang, H., Li, X. S., & Bing, L. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.49 · Cited by 608 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01855 · Cited by 540 · see citing works
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521 · see citing works
PandaGPT: One Model To Instruction-Follow Them AllPreprint
Su, Y., Lan, T., Li, H., Xu, J., Wang, Y., & Cai, D. (2023) · Tsinghua Interdisciplinary Workshop on Logic, Language and Meaning · doi:10.48550/arxiv.2305.16355 · Cited by 474 · see citing works
Grounding Action Descriptions in Videos
Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00207 · Cited by 471 · see citing works
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469 · see citing works
Long-Term Feature Banks for Detailed Video Understanding
Wu, C.-Y., Feichtenhofer, C., Fan, H., He, K., Krähenbühl, P., & Girshick, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00037 · Cited by 454 · see citing works
XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model
Cheng, H. K., & Schwing, A. G. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19815-1_37 · Cited by 449 · see citing works
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Maaz, M., Rasheed, H., Khan, S. H., & Khan, F. S. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.679 · Cited by 444 · see citing works
Video Question Answering via Gradually Refined Attention over Appearance and Motion
Xu, D., Zhao, Z., Xiao, J., Wu, F., Zhang, H., He, X., & Zhuang, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123427 · Cited by 438 · see citing works
Self-citationVideoChat: chat-centric video understanding
Li, K., He, Y., Wang, Y., Li, Y., Wang, W., Luo, P., Wang, L., Wang, L., & Qiao, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4321-9 · Cited by 426 · see citing works
Learning Dynamic Memory Networks for Object Tracking
Yang, T., & Chan, A. B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_10 · Cited by 382 · see citing works
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
Yu, Z., Xu, D., Yu, J., Yu, T., Zhao, Z., Zhuang, Y., & Tao, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019127 · Cited by 281 · cited 2× · see citing works
“The evaluation pipeline covers crucial metrics (including Correctness of Information , Detailed Orientation , Contextual Understanding , Temporal Understanding and Consistency ) and assigns relative scores to the generated predictions on a scale of 1-5.”
MethodMeMOT: Multi-Object Tracking with Memory
Cai, J., Xu, M., Li, W., Xiong, Y., Xia, W., Tu, Z., & Soatto, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00792 · Cited by 265 · see citing works
Otter: A Multi-Modal Model With In-Context Instruction Tuning
Li, B., Zhang, Y., Chen, L., Wang, J., Pu, F., Cahyono, J. A., Yang, J., Li, C., & Liu, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3571946 · Cited by 253 · see citing works
Self-citationCoherent Multi-sentence Video Description with Variable Level of Detail
Rohrbach, A., Rohrbach, M., Qiu, W., Friedrich, A., Pinkal, M., & Schiele, B. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_15 · Cited by 209 · see citing works
Kernelized Memory Network for Video Object Segmentation
Seong, H., Hyun, J., & Kim, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_38 · Cited by 204 · see citing works
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionPreprint
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., & Qiao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.16199 · Cited by 202 · see citing works
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
Wang, W., Chen, Z., Chen, X., Wu, J., Zhu, X., Zeng, G., Luo, P., Lu, T., Zhou, J., Qiao, Y., et al. (2023) · neural information processing systems · doi:10.52202/075280-2688 · Cited by 198 · see citing works
Script Data for Attribute-Based Recognition of Composite Activities
Rohrbach, M., Regneri, M., Andriluka, M., Amin, S., Pinkal, M., & Schiele, B. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_11 · Cited by 186 · see citing works
Recognizing Fine-Grained and Composite Activities Using Hand-Centric Features and Script Data
Rohrbach, M., Rohrbach, A., Regneri, M., Amin, S., Andriluka, M., Pinkal, M., & Schiele, B. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0851-8 · Cited by 182 · see citing works
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Wu, C.-Y., Li, Y., Mangalam, K., Fan, H., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01322 · Cited by 176 · see citing works
MovieNet: A Holistic Dataset for Movie Understanding
Huang, Q., Xiong, Y., Rao, A., Wang, J., & Lin, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_41 · Cited by 172 · see citing works
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking
Ma, C., Huang, J.-B., Yang, X., & Yang, M.-H. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1076-4 · Cited by 156 · see citing works
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Xu, L., Yang, J., Zheng, X., Li, K., Sun, X. W., et al. (2025) · neural information processing systems · doi:10.52202/085713-4899 · Cited by 148 · see citing works
Token Merging: Your ViT But Faster
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2022) · arXiv · doi:10.48550/arxiv.2210.09461 · Cited by 140 · see citing works
Temporal Aggregate Representations for Long-Range Video Understanding
Şener, F., Singhania, D., & Yao, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_10 · Cited by 139 · see citing works
Learning Position and Target Consistency for Memory-based Video Object Segmentation
Hu, L., Zhang, P., Zhang, B., Pan, P., Xu, Y., & Jin, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00413 · Cited by 129 · see citing works
Hierarchical Memory Matching Network for Video Object Segmentation
Seong, H., Oh, S. W., Lee, J., Lee, S., Lee, S., & Kim, E. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01265 · Cited by 129 · see citing works
Towards Long-Form Video Understanding
Wu, C.-Y., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00192 · Cited by 124 · see citing works
MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions
Soldan, M., Pardo, A., Alcázar, J. L., Heilbron, F. C., Zhao, C., Giancola, S., & Ghanem, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00497 · Cited by 94 · see citing works
Multiple-Target Tracking: A Role for Working Memory?
Allen, R., McGeorge, P., Pearson, D., & Milne, A. B. (2005) · Quarterly Journal of Experimental Psychology · doi:10.1080/02724980543000097 · Cited by 89 · see citing works
Zero-Shot Video Question Answering via Frozen Bidirectional Language ModelsPreprint
Yang, A., Miech, A., Šivic, J., Ivan Laptev, A., & Schmid, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.08155 · Cited by 87 · see citing works
Title Generation for User Generated Videos
Zeng, K.-H., Chen, T.-H., Niebles, J. C., & Sun, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_38 · Cited by 78 · see citing works
EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding
Mangalam, K., Akshulakov, R., & Malik, J. (2023) · neural information processing systems · doi:10.52202/075280-2004 · Cited by 77 · see citing works
MultiModal-GPT: a vision and language model for dialogue with humans
Gong, T., Lyu, C., Zhang, S., Wang, Y., Zheng, M., Zhao, Q., Liu, K., Zhang, W., Luo, P., & Chen, K. (2023) · arXiv · doi:10.48550/arxiv.2305.04790 · Cited by 70 · see citing works
Generic Event Boundary Detection: A Benchmark for Event Segmentation
Shou, M. Z., Lei, S. W., Wang, W., Ghadiyaram, D., & Feiszli, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00797 · Cited by 65 · see citing works
Memory-and-Anticipation Transformer for Online Action Understanding
Wang, J., Chen, G., Huang, Y., Wang, L., & Lu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01271 · Cited by 62 · see citing works
Notebaard, J. (2024) · doi:10.33540/2168 · Cited by 57 · see citing works
Generating Descriptions with Grounded and Co-referenced People
Rohrbach, A., Rohrbach, M., Tang, S., Oh, S. J., & Schiele, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.447 · Cited by 54 · see citing works
Hierarchical Video-Moment Retrieval and Step-Captioning
Zala, A., Cho, J., Kottur, S., Chen, X., Oğuz, B., Mehdad, Y., & Bansal, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02208 · Cited by 45 · see citing works
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 43 · see citing works
Self-citationAwad, G., Butt, A. A., Fiscus, J. G., Joy, D., Delgado, A., Mcclinton, W., Michel, M., Smeaton, A. F., Graham, Y., Kraaij, W., et al. (2017) · HAL · Cited by 42 · see citing works
(2011) · SpringerReference · doi:10.1007/springerreference_9081 · Cited by 38 · see citing works
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
Lyu, C., Wu, M., Wang, L., Huang, X., Liu, B., Du, Z., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2306.09093 · Cited by 38 · see citing works
Memory Network With Pixel-Level Spatio-Temporal Learning for Visual Object Tracking
Zhou, Z., Zhou, X., Chen, Z., Guo, P., Liu, Q., & Zhang, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3272319 · Cited by 27 · see citing works
Awad, G., Butt, A. A., Curtis, K., Lee, Y., Fiscus, J., Godil, A., Delgado, A., Zhang, J., Godard, E., Diduch, L. L., et al. (2019) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 26 · see citing works
Awad, G., Butt, A. A., Curtis, K., Fiscus, J., Godil, A., Lee, Y., Delgado, A., Zhang, J., Godard, E., Chocot, B., et al. (2021) · arXiv · doi:10.48550/arxiv.2104.13473 · Cited by 18 · see citing works
UMOTMA: Underwater multiple object tracking with memory aggregation
Hao, Z., Qiu, J., Zhang, H., Ren, G., & Liu, C. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.1071618 · Cited by 17 · see citing works
ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding SystemPreprint
Wang, J., Chen, D., Luo, C., Dai, X., Yuan, L., Wu, Z., & Jiang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.14407 · Cited by 9 · see citing works
MAVOT: Memory-Augmented Video Object TrackingPreprint
Liu, B., Wang, Y., Tai, Y., & Tang, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09414 · Cited by 7 · see citing works
Multi-Object Tracking with Spatial-Temporal Correlation Memory Networks
Xin, M., Sun, W., Li, K., & Hui, G. (2022) · 2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA) · doi:10.1109/cvidliccea56201.2022.9825193 · Cited by 2 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
3 of the 76 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- George Awad, Asad A Butt, Keith Curtis, Yooyoung Lee, Jonathan Fiscus, Afzad Godil, David Joy, Andrew Delgado, Alan F Smeaton, Yvette Graham, et al. Trecvid 2018: Benchmarking video activity detection, video captioning and matching, video storytelling linking and video search. In Proceedings of TRECVID 2018, 2018. 2
- Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 1728-1738, 2021. 2
- Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. CoRR, abs/2104.00650, 2021. 14 · arXiv
- Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, and Mike Zheng Shou. Mist: Multi-modal iterative spatial- temporal transformer for long-form video question answer- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 14773-14783, 2023. 2
- Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. 1, 2 · arXiv
- OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 1, 2 · arXiv
Cited by
Sorted by citations
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349 · see citing works
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02245 · Cited by 291 · see citing works
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Li, Y., Wang, C., & Jia, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_19 · Cited by 197 · see citing works
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
Ren, S., Yao, L., Li, S., Sun, X., & Hou, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01357 · Cited by 165 · see citing works
VideoAgent: Long-Form Video Understanding with Large Language Model as Agent
Wang, X., Zhang, Y., Zohar, O., & Yeung, S. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_4 · Cited by 97 · see citing works
Self-citationVideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 83 · see citing works
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 93 · see citing works
Li, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 72 · see citing works
VideoAgent: A Memory-Augmented Multimodal Agent for Video Understanding
Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z. F., & Li, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_5 · Cited by 67 · see citing works
LongVLM: Efficient Long Video Understanding via Large Language Models
Weng, Y., Han, M., He, H., Chang, X., & Zhuang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73414-4_26 · Cited by 63 · see citing works
3D Reconstruction with Spatial Memory
Wang, H., & Agapito, L. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00013 · Cited by 66 · see citing works
Streaming Dense Video Captioning
Zhou, X., Arnab, A., Buch, S., Shen, Y., Myers, A., Xiong, X., Nagrani, A., & Schmid, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01727 · Cited by 58 · see citing works
An Image Grid Can Be Worth a Video: Zero-Shot Video Question Answering Using a VLM
Kim, W., Choi, C., Lee, W., & Rhee, W. (2024) · IEEE Access · doi:10.1109/access.2024.3517625 · Cited by 50 · see citing works
Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding
Shu, Y., Liu, Z., Zhang, P., Qin, M., Zhou, J., Liang, Z., Huang, T., & Zhao, B. Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02436 · Cited by 59 · see citing works
M-LLM Based Video Frame Selection for Efficient Video Understanding
Hu, K., Gao, F., Nie, X., Peng, Z., Tran, S., Neiman, T., Wang, L., Shah, M., Hamid, R., Yin, B., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01279 · Cited by 34 · see citing works
Embodied Understanding of Driving Scenarios
Zhou, Y., Huang, L., Bu, Q., Zeng, J., Li, T., Qiu, H., Zhu, H., Guo, M., Qiao, Y., & Li, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_8 · Cited by 31 · see citing works
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration
Lin, Y., Lin, Z., Chen, H., Pan, P., Li, C., Chen, S., Wen, K., Jin, Y., Li, W., & Ding, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02084 · Cited by 27 · see citing works
Han, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., & Liu, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02438 · Cited by 22 · see citing works
Fu, H., Zhang, D. H., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., & Bai, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02302 · Cited by 37 · see citing works
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
Zhang, C., Lin, K., Yang, Z., Wang, J., Li, L., Lin, C.-C., Liu, Z., & Wang, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01295 · Cited by 24 · see citing works
AutoAD III: The Prequel - Back to the Pixels
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01720 · Cited by 24 · see citing works
DrVideo: Document Retrieval Based Long Video Understanding
Ma, Z., Gou, C., Shi, H., Sun, B., Shutao Li, A., Rezatofighi, H., & Cai, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01764 · Cited by 34 · see citing works
Qian, R., Ding, S., Dong, X., Zhang, P., Zang, Y., Cao, Y., Lin, D., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02239 · Cited by 26 · see citing works
Chen, S., Tian, Y., Zhang, K., Xing, Z., Lin, Y., & Zhu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_6 · Cited by 21 · see citing works
Hong, W., Cheng, Y., Yang, Z., Wang, W., Wang, L., Gu, X., Huang, S., Dong, Y., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00791 · Cited by 22 · see citing works
Koala: Key Frame-Conditioned Long Video-LLM
Tan, R., Sun, X., Hu, P., Wang, J.-H., Deilamsalehy, H., Plummer, B. A., Russell, B. C., & Saenko, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01289 · Cited by 19 · see citing works
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v1 · Cited by 18 · see citing works
Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review
Fang, Y., Miao, H., Liang, Y., Deng, L., Cui, Y., Zeng, X., Xia, Y., Zhao, Y. L., Pedersen, T. B., Jensen, C. N., et al. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3651536 · Cited by 16 · see citing works
TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos
Yao, L., Li, Y., Wei, Y., Li, L., Ren, S., Liu, Y., Ouyang, K., Wang, L., Li, S., Li, S., et al. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754839 · Cited by 22 · see citing works
Self-citationHA-Bins: Hierarchical Adaptive Bins for Robust Monocular Depth Estimation Across Multiple Datasets
Zhu, R., Song, Z., Liu, L., He, J., Zhang, T., & Zhang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3335316 · Cited by 15 · see citing works
Self-citationAbout Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 15 · see citing works
Video Question Answering with Procedural Programs
Choudhury, R., Niinuma, K., Kitani, K., & Jeni, L. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_18 · Cited by 14 · see citing works
See and Think: Embodied Agent in Virtual Environment
Zhao, Z., Chai, W., Wang, X., Li, B., Hao, S., Cao, S., Tian, Y., & Wang, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_11 · Cited by 21 · see citing works
Self-citationToward Long Video Understanding via Fine-Detailed Video Story Generation
You, Z., Wen, Z., Chen, Y., Li, X., Zeng, R., Wang, Y., & Tan, M. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3514820 · Cited by 11 · see citing works
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation
Nguyen, T.-T., Nguyen, P., Cothren, J. D., Yılmaz, A., & Luu, K. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02714 · Cited by 11 · see citing works
MambaMOT: State-Space Model as Motion Predictor for Multi-Object Tracking
Huang, H.-W., Yang, C.-Y., Chai, W., Jiang, Z., & Hwang, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890199 · Cited by 14 · see citing works
Self-citationJi, Z., An, P., & Zhao, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735762 · Cited by 10 · see citing works
Goldfish: Vision-Language Understanding of Arbitrarily Long Videos
Ataallah, K., Shen, X., Abdelrahman, E., Sleiman, E., Zhuge, M., Ding, J., Zhu, D., Schmidhuber, J., & Elhoseiny, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_15 · Cited by 9 · see citing works
Live: Learning Video LLM with Streaming Speech Transcription at Scale
Chen, J., Zeng, Z., Lin, Y., Li, W., Ma, Z., & Shou, M. Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02708 · Cited by 9 · see citing works
Elysium: Exploring Object-Level Perception in Videos via MLLM
Wang, H., Ye, Y., Wang, Y., Nie, Y., & Huang, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_10 · Cited by 8 · see citing works
DEEMO: De-identity Multimodal Emotion Recognition and Reasoning
Deng, L., Xing, B., Liu, X., Xia, B., Wen, B., & Kalviainen, H. A. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755411 · Cited by 8 · see citing works
VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges
Wang, Y., Song, Y., Xie, C., Liu, Y., & Zheng, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02240 · Cited by 8 · see citing works
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v2 · Cited by 8 · see citing works
ReWind: Understanding Long Videos with Instructed Learnable Memory
Diko, A., Wang, T., Swaileh, W., Sun, S., & Patras, I. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01282 · Cited by 7 · see citing works
Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check
Kuan, S.-Y., Cheng, J., Huang, H.-W., Chai, W., Yang, C.-Y., Latapie, H. M., Liu, G., Bing-Fei, W., & Hwang, J. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588514 · Cited by 7 · see citing works
Self-citationAMEGO: Active Memory from Long EGOcentric Videos
Goletto, G., Nagarajan, T., Averta, G., & Damen, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_6 · Cited by 7 · see citing works
Text-Conditioned Resampler For Long Form Video Understanding
Korbar, B., Xian, Y., Tonioni, A., Zisserman, A., & Tombari, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_16 · Cited by 9 · see citing works
MLLM-TA: Leveraging Multimodal Large Language Models for Precise Temporal Video Grounding
Liu, Y., Hou, H., Ma, F., Ni, S., & Yu, F. R. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3511426 · Cited by 6 · see citing works
Prospects and technology of embodied intelligent humanoid robots driven by AI large models
Kai, Z. E. N. G., Wang, Y., Tan, H., Fang, Q., Wang, Y., & Yuan, L. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0350 · Cited by 6 · see citing works
Kim, J., Kim, H., Lee, H., & Ro, Y. M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00318 · Cited by 6 · see citing works
SAMURAI: Motion-Aware Memory for Training-Free Visual Object Tracking With SAM 2
Yang, C.-Y., Huang, H.-W., Chai, W., Jiang, Z., & Hwang, J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3651835 · Cited by 52 · see citing works
Self-citationTimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation
You, L., Huang, W., Xie, X., Wei, X.-Y., Li, B., Lin, S., Li, Y., & Wang, C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755077 · Cited by 5 · see citing works
Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
Liu, J., Wang, Y., Ma, H., Wu, X., Ma, X., Wei, X., Jiao, J., Wu, E., & Hu, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02620-2 · Cited by 28 · see citing works
Sequential Affinity Learning for Video Restoration
Tian, Y., Chen, S., Liu, Y., Chai, W., Bai, J., Zou, W., Zhang, Y., Jiang, M., Chen, E., & Xue, C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611742 · Cited by 5 · see citing works
Self-citationLi, Z., Liu, Y., Yang, C., Zhou, Y., & Hu, S. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3590929 · Cited by 5 · see citing works
Hybrid-Level Instruction Injection for Video Token Compression in Multi-modal Large Language Models
Liu, Z., Xie, C.-W., Li, P., Zhao, L., Tang, L., Zheng, Y. P., Liu, C., & Xie, H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00801 · Cited by 4 · see citing works
UIL-AQA: Uncertainty-Aware Clip-Level Interpretable Action Quality Assessment
Dong, X., Liu, X., Li, W., Adeyemi-Ejeye, A. O. T., & Gilbert, A. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02638-6 · Cited by 4 · see citing works
StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognition
Ding, X., Wu, H. W., Yang, Y., Jiang, S., Zhang, Q., Bai, D., Chen, Z., & Cao, T. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01249 · Cited by 6 · see citing works
Zhuge, Y., Gong, S., Zhang, L., Xu, Q., Zhao, W., Zhan, J., & Lu, H. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3689427 · Cited by 4 · see citing works
A Survey on Fine-Grained Multimodal Large Language Models
Peng, Y., Wang, Z., Li, G., Zheng, X., Yin, S., & He, H. (2026) · Chinese Journal of Electronics · doi:10.23919/cje.2025.00.336 · Cited by 5 · see citing works
Movie Visual and Speech Analysis Through Multi-Modal LLM for Recommendation Systems
Qi, P. (2024) · IEEE Access · doi:10.1109/access.2024.3471568 · Cited by 4 · see citing works
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3 · see citing works
Hierarchical Event Memory for Accurate and Low-Latency Online Video Temporal Grounding
Zheng, M., Peng, Y., Sun, B., Yang, Y., & Liu, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02005 · Cited by 3 · see citing works
Efficient Transfer From Image-Based Large Multimodal Models to Video Tasks
Cao, S., Zhao, Z., Hao, S., Chai, W., Hwang, J., Wang, H., & Wang, G. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557692 · Cited by 3 · see citing works
Self-citationObject-Centric Cross-Modal Knowledge Reasoning for Future Event Prediction in Videos
Lai, C., Wang, H., Ge, W., & Xue, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3444895 · Cited by 3 · see citing works
Long Video Understanding With Learnable Retrieval in Video-Language Models
Xu, J., Lan, C., Xie, W., Chen, X., & Lu, Y. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3664996 · Cited by 3 · see citing works
Self-citationSeeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
Long, L., He, Y., Ye, W., Pan, Y., Lin, Y., Li, H., Zhao, J., & Li, W. (2025) · arXiv · doi:10.48550/arxiv.2508.09736 · Cited by 2 · see citing works
“We further compare our approach with three online video understanding frameworks: MovieChat [40], MA-LMM [12], and Flash-VStream [56].”
InfluentialCharacter-Aware Audio-Visual Subtitling in Context
Huh, J., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_21 · Cited by 2 · see citing works
Lai, C., Ge, W., & Xue, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3599038 · Cited by 2 · see citing works
REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding
Reza, S., Song, X., Yu, H., Lin, Z., Moghaddam, M., & Camps, O. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00244 · Cited by 2 · see citing works
Enhancing Long Video Understanding via Hierarchical Event-Based Memory
Cheng, D., Li, M., Liu, J., Guo, Y., Jiang, B., Liu, Q., Chen, X., & Zhao, B. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11210102 · Cited by 2 · see citing works
Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Liu, F., Wu, D., Chi, J., Cai, Y., Hung, Y., Yu, X., Li, H., Hu, H., Rao, Y., & Duan, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36839-3_19 · Cited by 2 · see citing works
Streaming Videollms for Real-Time Procedural Video Understanding
Chatterjee, D., Remelli, E., Song, Y., Tekin, B., Mittal, A., Bhatnagar, B., Camgöz, N. C., Hampali, S., Sauser, E. L., Ma, S., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02097 · Cited by 3 · see citing works
4D-Bench: Benchmarking Multi-Modal Large Language Models for 4D Object Understanding
Zhu, W., Li, B., Zheng, C., Mai, J., Chen, J., Jiang, L., Hamdi, A., Martinez, S. R., Lin, C., Elhoseiny, M., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01963 · Cited by 2 · see citing works
Training-free Dense Video Captioning with Large-scale Pretrained Models
Feng, Y., Yan, Z., Jia, Y., Chen, E. Q., & Qin, J. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1585-x · Cited by 2 · see citing works
Learning Video Context as Interleaved Multimodal Sequences
Lin, K. Q., Zhang, P., Gao, D., Xia, X., Chen, J., Gao, Z., Xie, J., Xiao, X., & Shou, M. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72967-6_21 · Cited by 3 · see citing works
It’s Just Another Day: Unique Video Captioning by Discriminitive Prompting
Perrett, T., Han, T., Damen, D., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_16 · Cited by 2 · see citing works
LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answering
Zhang, H., Dong, L., Liu, Y., Huang, Y., Wang, L., Wang, L., & Qiao, Y. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02555-8 · Cited by 10 · see citing works
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
Song, E., Chai, W., Xu, W., Xie, J., Liu, Y., & Wang, G. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00642 · Cited by 3 · see citing works
Self-citationVideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic VideosPreprint
Li, Z., Wu, X., Qin, Y., Shi, G., Du, H., Manocha, D., Zhou, T., & Boyd-Graber, J. L. (2025) · Qeios · doi:10.32388/bxc6x1 · Cited by 2 · see citing works
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 1 · see citing works
Self-citationLuo, Z., Wu, H., Li, D., Ma, J., Kankanhalli, M., & Li, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00792 · Cited by 6 · see citing works
Towards training-free long video understanding: methods, benchmarks, and open challenges
Liu, J., Wang, Y., Zhang, L., Wang, Y., Xu, S., Wang, L., Yan, J., Zhang, D., & Chen, X. (2025) · Vicinagearth. · doi:10.1007/s44336-025-00017-w · Cited by 1 · see citing works
GraphVideoAgent: Enhancing Long-form Video Understanding with Entity Relation Graphs
Chu, M., Li, Y., & Chua, T. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755537 · Cited by 3 · see citing works
FineQuest: Adaptive Knowledge-Assisted Sports Video Understanding via Agent-of-Thoughts Reasoning
Haodong, C., Huang, H., Yin, X., & Shao, D. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754885 · Cited by 1 · see citing works
Self-ReS: Self-Reflection in Large Vision-Language Models for Long Video Understanding
Pereira, J. M., Lopes, V., Semedo, D., & Neves, J. C. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209055 · Cited by 1 · see citing works
HLV-1K: A Large-scale Hour-Long Video Benchmark for Time-Specific Long Video Understanding
Zou, H., Luo, T., Xie, G., Zhang, V. X. J., Lv, F., Wang, G., Chen, J., Wang, Z., Zhang, H., & Zhang, H. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208965 · Cited by 1 · see citing works
Self-citationAttend and Replay: Efficient Action Understanding in Long Videos via Mechanistic Interpretability
Hou, P., Zhang, J., & Huang, D. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00688 · Cited by 1 · see citing works
Toward Scalable Video Narration: A Training-Free Approach Using Multimodal Large Language Models
Wu, T.-Y., Trigui, T., Sridhar, S. N., Bodas, A., & Tripathi, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00640 · Cited by 1 · see citing works
Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
Guan, Y., Yin, L., Liang, D., Ju, J., Luo, Z., Luan, J., Liu, Y., & Bai, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_5 · Cited by 4 · see citing works
LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
Liang, H., Cai, Q., Han, Z., Dong, H., Qiang, M., An, R., Xu, Q., Cui, B., & Zhang, W. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792099 · Cited by 4 · see citing works
See What You Need: Query-Aware Visual Intelligence Through Reasoning-Perception Loops
Dong, Z., Peng, B., Wang, Y., Liu, L., Dong, X., Cao, Y., & Wang, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462019 · Cited by 1 · see citing works
One Trajectory, One Token: Grounded Video Tokenization Via Panoptic Sub-Object Trajectory
Zheng, C., Zhang, J., Salehi, M., Gao, Z., Iyengar, V., Kobori, N., Kong, Q., & Krishna, R. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02150 · Cited by 1 · see citing works
Scalable Video Action Anticipation with Cross Linear Attentive Memory
Zhong, Z., Martín, M., Schneider, D., Lerch, D. J., Wu, C., Diederichs, F., Gall, J., & Beyerer, J. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00783 · Cited by 1 · see citing works
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
Wang, X., Wu, J., Lin, Z., Zhang, F., Zhang, D., & Nie, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3528394 · Cited by 1 · see citing works
Le, H. H., Nguyen, D. M. H., Bhatti, O. S., Kopácsi, L., Ngo, T. P., Nguyen, B. T., Barz, M., & Sonntag, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-94593-y · Cited by 1 · see citing works
Chappa, N. V. S. R., Dobbs, P. D., Raj, B., & Luu, K. (2024) · doi:10.36227/techrxiv.172296821.18796721/v1 · Cited by 1 · see citing works
CVBench: Benchmarking Cross-Video Synergies for Complex Multimodal ReasoningPreprint
Zhu, N., Dong, Y., Wang, T., Li, X., Deng, S., Wang, Y., Hong, Z., Geng, T., Niu, G., Huang, H., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6205842 · Cited by 1 · see citing works
MemoTiChat: A Memory-Augmented Time-Sensitive Model for Ultra-Long Video Understanding
Hou, J., Tao, T., Wang, J., Chen, Z., Ding, X., & Wang, K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649977
Learning optical flow from spiking camera with direction disassembly
Zhai, M., Jiang, M., Xiang, X., Ni, K., & Gao, H. (2025) · IET Image Processing · doi:10.1049/ipr2.13322
- Authors
- Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., Lu, Y., Hwang, J.-N., & Wang, G.
- Affiliations
- Zhejiang University (CN)
- University of Washington (US)
- Donghua University (CN)
- Microsoft Research Asia (China) (CN)
- Hong Kong University of Science and Technology (GZ)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 18221-18232
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402702917
- Cited by
- 176
- References
- 76
- Citations by year
- 2026: 82 · 2025: 60 · 2024: 32 · 2023: 2
- FWCI
- 35.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key R&D Program of China (2022ZD0162000)
- National Natural Science Foundation of China (62106219,62206046)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Keywords
- long video understanding, video foundation models, large language models, temporal relations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
