The DOI matches its Crossref record; 73 of the 76 references it cites were found in a registry or an index.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Open accessAbstract
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks. Yet, existing systems can only handle videos with very few frames. For long videos, the computation complexity, memory cost, and long-term temporal connection impose additional challenges. Taking advantage of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination with our specially designed memory mechanism, we propose the MovieChat to overcome these challenges. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1K benchmark with 1K long video and 14K manual annotations for validation of the effectiveness of our method. The code, models and data can be found in https://reself.github.io/MovieChat.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“By making plans and structured environment representations the readable and writable core of working memory, agents can maintain goal consistency and revise strategies robustly against perception failures (Song et al., 2023).”
From Memory in the Age of AI Agents (2025) · cites this paper as Song et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Self-citationLiu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,888
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741
Research works printed but not linked: 10
- 2023, Meet claude
- Awad, 2018, Trecvid 2018: Benchmarking video activity detection, video captioning and matching, video storytelling linking and video search
- 10.1109/ICCV48922.2021.00175 · DOI · Paper page
- Chiang, 2023, Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
- Driess, 2023, Palme: An embodied multimodal language model.
- 10.1007/springerreference_9081 · DOI
- 2021, Gpt3.5, 2021.
- 2023, Gpt-4 technical report.
- Su, 2023, Bert position encoding.
- Taori, 2023, Stanford alpaca: An instruction-following llama model
Cited by
Sorted by citations
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02245 · Cited by 291
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Li, Y., Wang, C., & Jia, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_19 · Cited by 403
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
Ren, S., Yao, L., Li, S., Sun, X., & Hou, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01357 · Cited by 277
VideoAgent: Long-Form Video Understanding with Large Language Model as Agent
Wang, X., Zhang, Y., Zohar, O., & Yeung, S. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_4 · Cited by 196
Self-citationVideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 124
VisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 229
Li, Z., Wu, X., Du, H., Liu, F., Nghiem, H. T., & Shi, G. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00147 · Cited by 91
VideoAgent: A Memory-Augmented Multimodal Agent for Video Understanding
Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z. F., & Li, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_5 · Cited by 114
LongVLM: Efficient Long Video Understanding via Large Language Models
Weng, Y., Han, M., He, H., Chang, X., & Zhuang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73414-4_26 · Cited by 108
3D Reconstruction with Spatial Memory
Wang, H., & Agapito, L. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00013 · Cited by 219
Streaming Dense Video Captioning
Zhou, X., Arnab, A., Buch, S., Shen, Y., Myers, A., Xiong, X., Nagrani, A., & Schmid, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01727 · Cited by 58
An Image Grid Can Be Worth a Video: Zero-Shot Video Question Answering Using a VLM
Kim, W., Choi, C., Lee, W., & Rhee, W. (2024) · IEEE Access · doi:10.1109/access.2024.3517625 · Cited by 101
Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding
Shu, Y., Liu, Z., Zhang, P., Qin, M., Zhou, J., Liang, Z., Huang, T., & Zhao, B. Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02436 · Cited by 148
M-LLM Based Video Frame Selection for Efficient Video Understanding
Hu, K., Gao, F., Nie, X., Peng, Z., Tran, S., Neiman, T., Wang, L., Shah, M., Hamid, R., Yin, B., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01279 · Cited by 34
Embodied Understanding of Driving Scenarios
Zhou, Y., Huang, L., Bu, Q., Zeng, J., Li, T., Qiu, H., Zhu, H., Guo, M., Qiao, Y., & Li, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_8 · Cited by 54
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration
Lin, Y., Lin, Z., Chen, H., Pan, P., Li, C., Chen, S., Wen, K., Jin, Y., Li, W., & Ding, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02084 · Cited by 31
Han, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., & Liu, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02438 · Cited by 22
Fu, H., Zhang, D. H., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., & Bai, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02302 · Cited by 123
MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
Zhang, C., Lin, K., Yang, Z., Wang, J., Li, L., Lin, C.-C., Liu, Z., & Wang, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01295 · Cited by 24
AutoAD III: The Prequel - Back to the Pixels
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01720 · Cited by 31
DrVideo: Document Retrieval Based Long Video Understanding
Ma, Z., Gou, C., Shi, H., Sun, B., Shutao Li, A., Rezatofighi, H., & Cai, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01764 · Cited by 34
Qian, R., Ding, S., Dong, X., Zhang, P., Zang, Y., Cao, Y., Lin, D., & Wang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02239 · Cited by 26
Chen, S., Tian, Y., Zhang, K., Xing, Z., Lin, Y., & Zhu, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_6 · Cited by 21
Hong, W., Cheng, Y., Yang, Z., Wang, W., Wang, L., Gu, X., Huang, S., Dong, Y., & Tang, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00791 · Cited by 22
Koala: Key Frame-Conditioned Long Video-LLM
Tan, R., Sun, X., Hu, P., Wang, J.-H., Deilamsalehy, H., Plummer, B. A., Russell, B. C., & Saenko, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01289 · Cited by 25
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v1 · Cited by 24
Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review
Fang, Y., Miao, H., Liang, Y., Deng, L., Cui, Y., Zeng, X., Xia, Y., Zhao, Y. L., Pedersen, T. B., Jensen, C. N., et al. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3651536 · Cited by 33
TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videos
Yao, L., Li, Y., Wei, Y., Li, L., Ren, S., Liu, Y., Ouyang, K., Wang, L., Li, S., Li, S., et al. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754839 · Cited by 22
Self-citationHA-Bins: Hierarchical Adaptive Bins for Robust Monocular Depth Estimation Across Multiple Datasets
Zhu, R., Song, Z., Liu, L., He, J., Zhang, T., & Zhang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3335316 · Cited by 15
Self-citationAbout Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 17
Video Question Answering with Procedural Programs
Choudhury, R., Niinuma, K., Kitani, K., & Jeni, L. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_18 · Cited by 19
See and Think: Embodied Agent in Virtual Environment
Zhao, Z., Chai, W., Wang, X., Li, B., Hao, S., Cao, S., Tian, Y., & Wang, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_11 · Cited by 41
Self-citationToward Long Video Understanding via Fine-Detailed Video Story Generation
You, Z., Wen, Z., Chen, Y., Li, X., Zeng, R., Wang, Y., & Tan, M. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3514820 · Cited by 11
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation
Nguyen, T.-T., Nguyen, P., Cothren, J. D., Yılmaz, A., & Luu, K. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02714 · Cited by 11
MambaMOT: State-Space Model as Motion Predictor for Multi-Object Tracking
Huang, H.-W., Yang, C.-Y., Chai, W., Jiang, Z., & Hwang, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890199 · Cited by 14
Self-citationJi, Z., An, P., & Zhao, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735762 · Cited by 10
Goldfish: Vision-Language Understanding of Arbitrarily Long Videos
Ataallah, K., Shen, X., Abdelrahman, E., Sleiman, E., Zhuge, M., Ding, J., Zhu, D., Schmidhuber, J., & Elhoseiny, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_15 · Cited by 35
Live: Learning Video LLM with Streaming Speech Transcription at Scale
Chen, J., Zeng, Z., Lin, Y., Li, W., Ma, Z., & Shou, M. Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02708 · Cited by 13
Elysium: Exploring Object-Level Perception in Videos via MLLM
Wang, H., Ye, Y., Wang, Y., Nie, Y., & Huang, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_10 · Cited by 18
DEEMO: De-identity Multimodal Emotion Recognition and Reasoning
Deng, L., Xing, B., Liu, X., Xia, B., Wen, B., & Kalviainen, H. A. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755411 · Cited by 26
VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridges
Wang, Y., Song, Y., Xie, C., Liu, Y., & Zheng, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02240 · Cited by 8
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v2 · Cited by 11
ReWind: Understanding Long Videos with Instructed Learnable Memory
Diko, A., Wang, T., Swaileh, W., Sun, S., & Patras, I. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01282 · Cited by 7
Boosting Online 3D Multi-Object Tracking through Camera-Radar Cross Check
Kuan, S.-Y., Cheng, J., Huang, H.-W., Chai, W., Yang, C.-Y., Latapie, H. M., Liu, G., Bing-Fei, W., & Hwang, J. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588514 · Cited by 7
Self-citationAMEGO: Active Memory from Long EGOcentric Videos
Goletto, G., Nagarajan, T., Averta, G., & Damen, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_6 · Cited by 7
Text-Conditioned Resampler For Long Form Video Understanding
Korbar, B., Xian, Y., Tonioni, A., Zisserman, A., & Tombari, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_16 · Cited by 14
MLLM-TA: Leveraging Multimodal Large Language Models for Precise Temporal Video Grounding
Liu, Y., Hou, H., Ma, F., Ni, S., & Yu, F. R. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3511426 · Cited by 6
Prospects and technology of embodied intelligent humanoid robots driven by AI large models
Kai, Z. E. N. G., Wang, Y., Tan, H., Fang, Q., Wang, Y., & Yuan, L. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0350 · Cited by 6
Kim, J., Kim, H., Lee, H., & Ro, Y. M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00318 · Cited by 6
SAMURAI: Motion-Aware Memory for Training-Free Visual Object Tracking With SAM 2
Yang, C.-Y., Huang, H.-W., Chai, W., Jiang, Z., & Hwang, J. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3651835 · Cited by 136
Self-citationTimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation
You, L., Huang, W., Xie, X., Wei, X.-Y., Li, B., Lin, S., Li, Y., & Wang, C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755077 · Cited by 10
Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input
Liu, J., Wang, Y., Ma, H., Wu, X., Ma, X., Wei, X., Jiao, J., Wu, E., & Hu, J. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02620-2 · Cited by 132
Sequential Affinity Learning for Video Restoration
Tian, Y., Chen, S., Liu, Y., Chai, W., Bai, J., Zou, W., Zhang, Y., Jiang, M., Chen, E., & Xue, C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611742 · Cited by 5
Self-citationLi, Z., Liu, Y., Yang, C., Zhou, Y., & Hu, S. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3590929 · Cited by 5
Hybrid-Level Instruction Injection for Video Token Compression in Multi-modal Large Language Models
Liu, Z., Xie, C.-W., Li, P., Zhao, L., Tang, L., Zheng, Y. P., Liu, C., & Xie, H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00801 · Cited by 13
UIL-AQA: Uncertainty-Aware Clip-Level Interpretable Action Quality Assessment
Dong, X., Liu, X., Li, W., Adeyemi-Ejeye, A. O. T., & Gilbert, A. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02638-6 · Cited by 4
StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognition
Ding, X., Wu, H. W., Yang, Y., Jiang, S., Zhang, Q., Bai, D., Chen, Z., & Cao, T. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01249 · Cited by 6
Zhuge, Y., Gong, S., Zhang, L., Xu, Q., Zhao, W., Zhan, J., & Lu, H. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3689427 · Cited by 5
A Survey on Fine-Grained Multimodal Large Language Models
Peng, Y., Wang, Z., Li, G., Zheng, X., Yin, S., & He, H. (2026) · Chinese Journal of Electronics · doi:10.23919/cje.2025.00.336 · Cited by 10
Movie Visual and Speech Analysis Through Multi-Modal LLM for Recommendation Systems
Qi, P. (2024) · IEEE Access · doi:10.1109/access.2024.3471568 · Cited by 7
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3
Hierarchical Event Memory for Accurate and Low-Latency Online Video Temporal Grounding
Zheng, M., Peng, Y., Sun, B., Yang, Y., & Liu, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02005 · Cited by 3
Efficient Transfer From Image-Based Large Multimodal Models to Video Tasks
Cao, S., Zhao, Z., Hao, S., Chai, W., Hwang, J., Wang, H., & Wang, G. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557692 · Cited by 3
Self-citationObject-Centric Cross-Modal Knowledge Reasoning for Future Event Prediction in Videos
Lai, C., Wang, H., Ge, W., & Xue, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3444895 · Cited by 4
Long Video Understanding With Learnable Retrieval in Video-Language Models
Xu, J., Lan, C., Xie, W., Chen, X., & Lu, Y. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3664996 · Cited by 29
Self-citationSeeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
Long, L., He, Y., Ye, W., Pan, Y., Lin, Y., Li, H., Zhao, J., & Li, W. (2025) · arXiv · doi:10.48550/arxiv.2508.09736 · Cited by 102
“We further compare our approach with three online video understanding frameworks: MovieChat [40], MA-LMM [12], and Flash-VStream [56].”
InfluentialCharacter-Aware Audio-Visual Subtitling in Context
Huh, J., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_21 · Cited by 3
Lai, C., Ge, W., & Xue, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3599038 · Cited by 2
REEF: Relevance-Aware and Efficient LLM Adapter for Video Understanding
Reza, S., Song, X., Yu, H., Lin, Z., Moghaddam, M., & Camps, O. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00244 · Cited by 2
Enhancing Long Video Understanding via Hierarchical Event-Based Memory
Cheng, D., Li, M., Liu, J., Guo, Y., Jiang, B., Liu, Q., Chen, X., & Zhao, B. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11210102 · Cited by 2
Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training
Liu, F., Wu, D., Chi, J., Cai, Y., Hung, Y., Yu, X., Li, H., Hu, H., Rao, Y., & Duan, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36839-3_19 · Cited by 12
Streaming Videollms for Real-Time Procedural Video Understanding
Chatterjee, D., Remelli, E., Song, Y., Tekin, B., Mittal, A., Bhatnagar, B., Camgöz, N. C., Hampali, S., Sauser, E. L., Ma, S., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02097 · Cited by 3
4D-Bench: Benchmarking Multi-Modal Large Language Models for 4D Object Understanding
Zhu, W., Li, B., Zheng, C., Mai, J., Chen, J., Jiang, L., Hamdi, A., Martinez, S. R., Lin, C., Elhoseiny, M., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01963 · Cited by 5
Training-free Dense Video Captioning with Large-scale Pretrained Models
Feng, Y., Yan, Z., Jia, Y., Chen, E. Q., & Qin, J. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1585-x · Cited by 2
Learning Video Context as Interleaved Multimodal Sequences
Lin, K. Q., Zhang, P., Gao, D., Xia, X., Chen, J., Gao, Z., Xie, J., Xiao, X., & Shou, M. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72967-6_21 · Cited by 5
It’s Just Another Day: Unique Video Captioning by Discriminitive Prompting
Perrett, T., Han, T., Damen, D., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_16 · Cited by 2
LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answering
Zhang, H., Dong, L., Liu, Y., Huang, Y., Wang, L., Wang, L., & Qiao, Y. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02555-8 · Cited by 48
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark
Song, E., Chai, W., Xu, W., Xie, J., Liu, Y., & Wang, G. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00642 · Cited by 3
Self-citationVideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic VideosPreprint
Li, Z., Wu, X., Qin, Y., Shi, G., Du, H., Manocha, D., Zhou, T., & Boyd-Graber, J. L. (2025) · Qeios · doi:10.32388/bxc6x1 · Cited by 2
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 389
Self-citationLuo, Z., Wu, H., Li, D., Ma, J., Kankanhalli, M., & Li, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00792 · Cited by 6
Towards training-free long video understanding: methods, benchmarks, and open challenges
Liu, J., Wang, Y., Zhang, L., Wang, Y., Xu, S., Wang, L., Yan, J., Zhang, D., & Chen, X. (2025) · Vicinagearth. · doi:10.1007/s44336-025-00017-w · Cited by 2
GraphVideoAgent: Enhancing Long-form Video Understanding with Entity Relation Graphs
Chu, M., Li, Y., & Chua, T. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755537 · Cited by 3
FineQuest: Adaptive Knowledge-Assisted Sports Video Understanding via Agent-of-Thoughts Reasoning
Haodong, C., Huang, H., Yin, X., & Shao, D. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754885 · Cited by 1
Self-ReS: Self-Reflection in Large Vision-Language Models for Long Video Understanding
Pereira, J. M., Lopes, V., Semedo, D., & Neves, J. C. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209055 · Cited by 1
HLV-1K: A Large-scale Hour-Long Video Benchmark for Time-Specific Long Video Understanding
Zou, H., Luo, T., Xie, G., Zhang, V. X. J., Lv, F., Wang, G., Chen, J., Wang, Z., Zhang, H., & Zhang, H. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11208965 · Cited by 1
Self-citationAttend and Replay: Efficient Action Understanding in Long Videos via Mechanistic Interpretability
Hou, P., Zhang, J., & Huang, D. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00688 · Cited by 1
Toward Scalable Video Narration: A Training-Free Approach Using Multimodal Large Language Models
Wu, T.-Y., Trigui, T., Sridhar, S. N., Bodas, A., & Tripathi, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00640 · Cited by 1
Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously
Guan, Y., Yin, L., Liang, D., Ju, J., Luo, Z., Luan, J., Liu, Y., & Bai, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_5 · Cited by 4
LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
Liang, H., Cai, Q., Han, Z., Dong, H., Qiang, M., An, R., Xu, Q., Cui, B., & Zhang, W. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792099 · Cited by 17
See What You Need: Query-Aware Visual Intelligence Through Reasoning-Perception Loops
Dong, Z., Peng, B., Wang, Y., Liu, L., Dong, X., Cao, Y., & Wang, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462019 · Cited by 1
One Trajectory, One Token: Grounded Video Tokenization Via Panoptic Sub-Object Trajectory
Zheng, C., Zhang, J., Salehi, M., Gao, Z., Iyengar, V., Kobori, N., Kong, Q., & Krishna, R. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02150 · Cited by 4
Scalable Video Action Anticipation with Cross Linear Attentive Memory
Zhong, Z., Martín, M., Schneider, D., Lerch, D. J., Wu, C., Diederichs, F., Gall, J., & Beyerer, J. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00783 · Cited by 2
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding
Wang, X., Wu, J., Lin, Z., Zhang, F., Zhang, D., & Nie, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3528394 · Cited by 1
Le, H. H., Nguyen, D. M. H., Bhatti, O. S., Kopácsi, L., Ngo, T. P., Nguyen, B. T., Barz, M., & Sonntag, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-94593-y · Cited by 1
Chappa, N. V. S. R., Dobbs, P. D., Raj, B., & Luu, K. (2024) · doi:10.36227/techrxiv.172296821.18796721/v1 · Cited by 1
CVBench: Benchmarking Cross-Video Synergies for Complex Multimodal ReasoningPreprint
Zhu, N., Dong, Y., Wang, T., Li, X., Deng, S., Wang, Y., Hong, Z., Geng, T., Niu, G., Huang, H., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6205842 · Cited by 1
MemoTiChat: A Memory-Augmented Time-Sensitive Model for Ultra-Long Video Understanding
Hou, J., Tao, T., Wang, J., Chen, Z., Ding, X., & Wang, K. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649977
Learning optical flow from spiking camera with direction disassembly
Zhai, M., Jiang, M., Xiang, X., Ni, K., & Gao, H. (2025) · IET Image Processing · doi:10.1049/ipr2.13322
Metadata
- Authors
- Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., Lu, Y., Hwang, J.-N., & Wang, G.
- Affiliations
- Zhejiang University (CN)
- University of Washington (US)
- Donghua University (CN)
- Microsoft Research Asia (China) (CN)
- Hong Kong University of Science and Technology (GZ)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 18221-18232
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402702917
- Cited by
- 176
- References
- 76
- Citations by year
- 2026: 82 · 2025: 60 · 2024: 32
- FWCI
- 35.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key R&D Program of China (2022ZD0162000)
- National Natural Science Foundation of China (62106219,62206046)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Keywords
- long video understanding, video foundation models, large language models, temporal relations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
