The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
MLVU: Benchmarking Multi-task Long Video Understanding
Open accessAbstract
The evaluation of Long Video Understanding (LVU) performance poses an important but challenging research problem. Despite previous efforts, the existing video understanding benchmarks are severely constrained by several issues, especially the insufficient lengths of videos, a lack of diversity in video types and evaluation tasks, and the inappropriateness for evaluating LVU performances. To address the above problems, we propose a new benchmark called MLVU (Multitask Long Video Understanding Benchmark) for the comprehensive and in-depth evaluation of LVU. MLVU presents the following critical values: 1) The substantial and flexible extension of video lengths, which enables the benchmark to evaluate LVU performance across a wide range of durations. 2) The inclusion of various video genres, such as movies, surveillance, egocentric videos, and cartoons, reflects the models’ LVU performances in different scenarios. 3) The development of diversified evaluation tasks, which enables a comprehensive examination of MLLMs’ key abilities in long-video understanding. The empirical study with 23 latest MLLMs reveals significant room for improvement in today’s technique, as all existing methods struggle with most of the evaluation tasks and exhibit severe performance degradation when handling longer videos. Additionally, it suggests that factors such as context length, image-understanding ability, and the choice of LLM backbone can play critical roles in future advancements. We anticipate that MLVU will advance the research of LVU by providing a comprehensive and in-depth analysis of MLLMs. The code and dataset can be accessed from https://github.com/JUNJIE99/MLVU.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most existing LVQA benchmarks [2, 11, 50, 62] mainly focus on visual understanding, such as action recognition and spatial/temporal perception, leaving a gap in evaluating higher-level cognitive abilities that rely on long-term memory and are crucial for real-world agents, such as understanding persons, extracting general knowledge, and performing cross-modal reasoning.”
From Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory (2025) · cites this paper as [62]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 8
- [6][6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020. 1 -
- [15][15] Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18995-19012, 2022. 1, 3, 4 -
- [34][34] Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very longform video language understanding. Advances in Neural Information Processing Systems, 36, 2023. 1, 2, 3 -
- [48][48] Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track (Round 2), 2021. 3 -
- [52][52] Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5288-5296, 2016. 1, 2, 3 -
- [55][55] Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. Clevrer: Collision events for video representation and reasoning. In International Conference on Learning Representations, 2019. 4 -
- [56][56] Zhou Yu, Dejing Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. Activitynet-qa: A dataset for understanding complex web videos via question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9127-9134, 2019. 3 -
- [63][63] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing visionlanguage understanding with advanced large language models. In The Twelfth International Conference on Learning Representations, 2023. 2
Cited by
Sorted by citations
Items 401–454 of 454
LongEgoRefer: A Benchmark for Long-Form Egocentric Video Referring Expression Comprehension
Kato, S., Miyanishi, T., Kurita, S., Ukai, M., Inoue, N., & Chu, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37261-1_18
DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video DifferencesPreprint
Yang, Y., Long, Y., Wen, B., Yang, F., Gao, T., Li, H., & Yang, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.02551
Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QAPreprint
Zhang, C., Wang, Z., Pan, Y., Oguntola, O., Wagh, P., Wu, Q., Wakaki, H., Bansal, M., & Bertasius, G. (2026) · arXiv (Cornell University)
Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
Song, J., Kim, W., & Kong, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37447-9_36
SparseVLM+: Visual Token Sparsification With Improved Text-Visual Attention Pattern
Zhang, Y., Ma, J., Zhang, Q., Fan, C.-K., Zheng, W., Cheng, K., Lu, J., & Zhang, S. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3718245
Reading Between the Frames of Multimodal DrivelologyPreprint
Wang, Y., Ma, Y., Chang, Z. Y., Loakman, T., Chen, C.-L., Hsiao, C.-Y., Jin, Y., Liu, Y., Villavicencio, A., Xiao, C., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.04939
VideoExplorer: Advancing Long-Horizon Video Understanding via Hierarchical Orchestration
Yuan, H., Liu, Z., Zhou, J., Qian, H., Shu, Y., Sebe, N., Wen, J.-R., & Dou, Z. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817705
Self-citationVMDT: Decoding the Trustworthiness of Video Foundation Models
Potter, Y., Wang, Z., Crispino, N., Montgomery, K., Xiong, A., Chang, E., Pinto, F., Chen, Y., Gupta, R., Ziyadi, M., et al. (2025) · neural information processing systems · doi:10.52202/085713-5368
EventLens: Event-Aware Hierarchical Evidence Construction for Long-Video Understanding
Zhu, Z., Dong, X. M., Xu, Y., Liu, Y., Yu, Z., Liu, L., & Zhang, H. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3723106
Speaking ITM’s Language: Query Reformulation and Temporal-MMR for Long-Video QA
Han, S., Vu, T., & Kim, J. (2026) · IEEE Access · doi:10.1109/access.2026.3725764
LongVU-TTT: Causal Test-Time Training for Visual Resampling in Long Video UnderstandingPreprint
Ahmed, M., Abdulah, S., Ruwase, O., Jacobs, S. A., Bode, M., & Elhoseiny, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.25729
Training-Free Temporal Abstraction for General Video UnderstandingPreprint
Casanova, E., Brodjian, S., & Perona, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.27929
Multi-Agent Self-Improving Reinforcement Learning for Video ReasoningPreprint
Zhang, M., Dang, J., Yang, M., Wang, B., Hu, B., & Chua, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.28675
Zhao, W., Liu, A., & Gao, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7412292
Huang, Z., Lai, M., Liu, S., Zhang, Y., & Wang, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37356-4_9
VI-Bench: Benchmarking Prompt Inversion from AIGC VideosPreprint
Xie, W., Zhao, R., Li, K., Liu, X., Zhang, B., Liu, Z., Hou, X., & Gong, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.08079
Self-citationEatVid-Bench: A Multimodal Fine-Grained Eating Behavior Video Dataset
Zheng, X., Xu, Z., Huang, G., Li, Z., & Yang, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_22
Benchmarking MLLMs on Mistake Recognition and Explanation in Single-Step Components of Cooking
Takashige, S., Hashimoto, A., & Satoh, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37255-0_36
CTEPM: Continuous-Time Event Process Memory for Long-Video Language Models
Liu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37490-5_25
H$$^2$$SVC: Head-Aware Heterogeneous Streaming Video Cache for Online Video Understanding
Li, H., Zhang, X., Dai, W., Wang, Y., Peng, X., He, F., Xu, H., Zheng, Z., Li, C., Zou, J., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37174-4_26
Query-aligned video frame selection for long video understandingPreprint
Islam, M. S., Sarkar, D., & Liang, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.31668
DIVE-Bench: Codec-Style High-FPS Video Understanding
Zhang, H., Chai, W., He, S., Li, A., & Fu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36987-1_23
Towards Effective Long Video Understanding: Dynamic MAS Construction via Meta-agent
Huang, J., Zhang, L., Li, Y., Xu, X., Chen, S., Liu, J., Kong, M., & Zhu, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36987-1_30
SYNCR: Diagnosing and Learning Cross-Video Reasoning from SimulationPreprint
Ghazanfari, S., Garg, S., Krishnamurthy, P., & Khorrami, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.37918
STVFocus: Query-Guided Spatio-Temporal Visual Focusing for Video LLMs
Kong, T., Park, M., & Ahn, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37135-5_18
GTR: Guide-Then-Refine Token Compression for Training-Free Acceleration of Video-LLMs
Zhou, S., Feng, W., Qi, Z., Wang, Z., Ju, L., An, X., Harandi, M. T., & Ge, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37258-1_27
Guan, H., Li, S., & Pei, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.00757
Efficient visual learning and reasoning system with spurious and redundant data on edge computing
Ling, T. (n.d.) · PolyU Institutional Research Archive (Hong Kong Polytechnic University)
Beyond Single Videos: Benchmarking and Active Evidence Seeking for E-Commerce Cross-Video ReasoningPreprint
Zhao, J., Hu, Y., Wu, L., Xu, J., & Zheng, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.03099
Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMsPreprint
Seo, H., Lee, B. H., Kim, M., Mah, D., Lee, J., & Chun, S. Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04580
HippoMM: Hippocampal-Inspired Multimodal Memory for Long Audiovisual Event Understanding
Lin, Y., Zhang, J., Wang, Q., Ye, H., Fu, Y., Liu, Y., Li, H. H., & Chen, Y. (2026) · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · doi:10.1109/cvprf71909.2026.00573 · Cited by 1
EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video UnderstandingPreprint
Qin, Y., Wang, J., Li, Y., & Zhu, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.08674
OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual CaptioningPreprint
Yang, Z., Tao, J., Chen, R., Yang, Z., Yuan, Y., Zhao, X., Auden, Wang, K., Shao, S., Wang, B., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.12458
DataVista: Diagnosing Multimodal LLMs on Data Video UnderstandingPreprint
Xie, Y., Wang, Z., Zhu, J., Tang, Y., Shen, Z., Chen, Y., & Luo, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.11993
WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMsPreprint
Ali, A., Zoabi, T., Wolf, L., & Baskin, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.17355
Video-Arena: A Scalable Agentic Video Benchmark for Spatiotemporal CapabilityPreprint
Zhang, K., Chen, Y., Yang, Z., Su, Y., Wu, X., Chen, Z., Luo, R., Tan, T., Zheng, Y., & Li, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14158
Wang, H., Wang, M., Zhao, J., Chen, H., Chen, L., & 乔竹风 (2026) · Applied Sciences · doi:10.3390/app162010327
Execution Before Memory: Diagnosing Memory Failures in Video World ModelsPreprint
Jin, W., Yu, J., Xu, S., Niu, Y., & Liu, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.23542
From Label Priors to Task Evidence: Efficient Long-Video Frame Selection via Bayesian GRPOPreprint
Chang, S., Gao, B., Liu, Q., Ma, Q., Miao, Y., Zhao, H., Liao, Z., Zhang, X., Li, J., & Niu, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.25418
LexSign: Learning Sign Language from Lexical Descriptions
Min, Y., Zou, H., Jiao, P., & Chen, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-03075-9
OpenHome: From Perception to Insight in Smart-Home Camera VideosPreprint
Wang, X., Rodrigues, N., Guo, P., Fang, Y., Beussman, K., Wang, S., Wang, J., Li, L., Xu, S., Cheng, Z., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.31986
Joint Latent Reasoning and Observation for Agentic Video UnderstandingPreprint
Liu, J., Yuan, H., Wu, Y., Zhang, Q., Chen, L., & Wang, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.35229
StreamVibe: Proactive Video Agents for Evolving User IntentPreprint
Liu, C., Lai, Y., Yang, Y., Liu, Y., Hou, J., Cao, Y., Wang, H., Fan, R., Wang, L., Liu, S., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.34662
Diagnosing and Correcting Bias in MLLM for Long Video UnderstandingPreprint
Liang, X., Sun, J., Zhang, H., Niu, Y., Zhao, H., & Ma, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.36210
Hong, S., Kim, J., & Kang, U. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1087
VISA: Video Skeleton-Aware Efficient Frame Selection for Long Video Understanding
Zhu, Q., Wen, Z., Zhang, T., Zhang, M., Ye, S., Jin, D., Chen, S., Tan, X., Zheng, Q., & Jianyang (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1142
LF-VideoQA: Evaluating Long-Video Understanding with Long-Form Theory-of-Mind QuestionsPreprint
Shaar, S., Thymes, B., Aldaye, M., Cardie, C., & Hariharan, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.00383
Selecting or Solving? What Best-of-N Verification Delivers in Video ReasoningPreprint
Ma, M. Q., Qu, Y., Salakhutdinov, R., Morency, L., Tomasello, P., & Pino, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.00507
Video * Bench: Benchmarking Fine-grained Video Perception with "Think with Videos"
Chen, R., Fan, Z., Shi, Y., Song, E., Chai, W., Guan, T., Jiang, S., Luo, R., Zhang, Y., Bai, S., et al. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3836293
Self-citationZhang, L. (2026) · Research Square · doi:10.21203/rs.3.rs-11412390/v1
NexusQA: Spatio-Temporal Interlacement Grounded VideoQA
Ye, X., Chun, W., Chen, J., Zhou, S., Wang, R., Yan, Z., Jia, B., Li, B., Song, Y., & Bu, J. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3835487
Question-in-the-Loop: Interleaved Query Tokens as Visual Summarizers for Video LLMs
Kang, D., Niu, Z., Lyu, J., Wang, H., Shen, L., Wang, J., & Qin, X. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3835485
APT: Alignment-Preserving Video Tokenizer for Non-Destructive Image-to-Video Adaptation
Gao, L., Zhong, Y., Li, Y., Tan, H., Hu, J., Kang, Z., Gao, Z., & Wei, X. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3835748
SemStream: Semantic Boundary-Aware Memory for Streaming Video Understanding
Tan, Z., Kuai, M., Lv, Z., Fei, H., & Ji, W. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3835466
Metadata
- Authors
- Zhou, J., Shu, Y., Zhao, B., Wu, B., Liang, Z., Xiao, S., Qin, M., Yang, X., Xiong, Y., Zhang, B., Huang, T., & Liu, Z.
- Affiliations
- Beijing University of Posts and Telecommunications (CN)
- Beijing Academy of Artificial Intelligence (CN)
- Shanghai Jiao Tong University (CN)
- Zhejiang University (CN)
- ORCID
- Venue
- 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2025-06-10
- Pages
- 13691-13701
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4413144845
- Cited by
- 44
- References
- 0
- Citations by year
- 2026: 37 · 2025: 6
- FWCI
- 32.37
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Keywords
- long video understanding, benchmark, multi-task evaluation, multimodal large language models, video length, video genres, surveillance video, egocentric video, animated cartoons, context length, image understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
