The DOI matches its Crossref record; 73 of the 76 references it cites were found in a registry or an index.
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Open accessAbstract
Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks. Yet, existing systems can only handle videos with very few frames. For long videos, the computation complexity, memory cost, and long-term temporal connection impose additional challenges. Taking advantage of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination with our specially designed memory mechanism, we propose the MovieChat to overcome these challenges. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1K benchmark with 1K long video and 14K manual annotations for validation of the effectiveness of our method. The code, models and data can be found in https://reself.github.io/MovieChat.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“By making plans and structured environment representations the readable and writable core of working memory, agents can maintain goal consistency and revise strategies robustly against perception failures (Song et al., 2023).”
From Memory in the Age of AI Agents (2025) · cites this paper as Song et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Human Memory: A Proposed System and its Control Processes
Atkinson, R. C., & Shiffrin, R. M. (1968) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60422-3 · Cited by 7,216
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Self-citationLiu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,888
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction TuningPreprint
Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, P., & Hoi, S. C. H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06500 · Cited by 1,372
mPLUG-Owl: Modularization Empowers Large Language Models with MultimodalityPreprint
Ye, Q.-H., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang, J., Hu, A., Shi, P.-C., Shi, Y., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2304.14178 · Cited by 1,318
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Maaz, M., Rasheed, H., Khan, S. H., & Khan, F. S. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.679 · Cited by 1,238
Birch, A., & Osborne, M. (2011) · The Association for Computational Linguistics · Cited by 1,196
VideoChat: chat-centric video understanding
Li, K., He, Y., Wang, Y., Li, Y., Wang, W., Luo, P., Wang, L., Wang, L., & Qiao, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4321-9 · Cited by 1,181
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init AttentionPreprint
Zhang, R., Han, J., Liu, C., Gao, P., Zhou, A., Hu, X., Yan, S., Lu, P., Li, H., & Qiao, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.16199 · Cited by 957
Collecting Highly Parallel Data for Paraphrase Evaluation
Chen, D. J., & Dolan, W. B. (2011) · Cited by 898
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Zhang, H., Li, X. S., & Bing, L. (2023) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2023.emnlp-demo.49 · Cited by 744
Squire, L. R., Genzel, L., Wixted, J. T., & Morris, R. G. (2015) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a021766 · Cited by 700 · cited 1×
“There are some prior works exploring memory models [56] in various vision tasks in videos, such as video object segmentation (VOS) [17,27,52,53], multi-object tracking (MOT) [2,13,26,69], visual object tracking (VOT) [35, 39, 73, 80], and action understanding [64].”
BackgroundLLaMA-Adapter V2: Parameter-Efficient Visual Instruction ModelPreprint
Gao, P., Han, J., Zhang, R., Lin, Z., Geng, S., Zhou, A., Zhang, W., Lu, P., He, C., Yue, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.15010 · Cited by 656
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01855 · Cited by 655
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591
Otter: A Multi-Modal Model With In-Context Instruction Tuning
Li, B., Zhang, Y., Chen, L., Wang, J., Pu, F., Cahyono, J. A., Yang, J., Li, C., & Liu, Z. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3571946 · Cited by 571
Self-citationA database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Grounding Action Descriptions in Videos
Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00207 · Cited by 471
Video Question Answering via Gradually Refined Attention over Appearance and Motion
Xu, D., Zhao, Z., Xiao, J., Wu, F., Zhang, H., He, X., & Zhuang, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123427 · Cited by 471
Self-citationMiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Zhu, D., Chen, J., Shen, X., Li, X., & Elhoseiny, M. (2023) · arXiv · doi:10.48550/arxiv.2304.10592 · Cited by 469
Long-Term Feature Banks for Detailed Video Understanding
Wu, C.-Y., Feichtenhofer, C., Fan, H., He, K., Krähenbühl, P., & Girshick, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00037 · Cited by 465
XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model
Cheng, H. K., & Schwing, A. G. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19815-1_37 · Cited by 449
Learning Dynamic Memory Networks for Object Tracking
Yang, T., & Chan, A. B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_10 · Cited by 382
PandaGPT: One Model To Instruction-Follow Them AllPreprint
Su, Y., Lan, T., Li, H., Xu, J., Wang, Y., & Cai, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.16355 · Cited by 379
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
Wang, W., Chen, Z., Chen, X., Wu, J., Zhu, X., Zeng, G., Luo, P., Lu, T., Zhou, J., Qiao, Y., et al. (2023) · neural information processing systems · doi:10.52202/075280-2688 · Cited by 368
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
Yu, Z., Xu, D., Yu, J., Yu, T., Zhao, Z., Zhuang, Y., & Tao, D. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019127 · Cited by 281 · cited 2×
“The evaluation pipeline covers crucial metrics (including Correctness of Information , Detailed Orientation , Contextual Understanding , Temporal Understanding and Consistency ) and assigns relative scores to the generated predictions on a scale of 1-5.”
MethodMeMOT: Multi-Object Tracking with Memory
Cai, J., Xu, M., Li, W., Xiong, Y., Xia, W., Tu, Z., & Soatto, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00792 · Cited by 265
Coherent Multi-sentence Video Description with Variable Level of Detail
Rohrbach, A., Rohrbach, M., Qiu, W., Friedrich, A., Pinkal, M., & Schiele, B. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_15 · Cited by 209
Kernelized Memory Network for Video Object Segmentation
Seong, H., Hyun, J., & Kim, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_38 · Cited by 204
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Wu, C.-Y., Li, Y., Mangalam, K., Fan, H., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01322 · Cited by 201
MovieNet: A Holistic Dataset for Movie Understanding
Huang, Q., Xiong, Y., Rao, A., Wang, J., & Lin, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_41 · Cited by 196
Script Data for Attribute-Based Recognition of Composite Activities
Rohrbach, M., Regneri, M., Andriluka, M., Amin, S., Pinkal, M., & Schiele, B. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_11 · Cited by 186
Recognizing Fine-Grained and Composite Activities Using Hand-Centric Features and Script Data
Rohrbach, M., Rohrbach, A., Regneri, M., Amin, S., Andriluka, M., Pinkal, M., & Schiele, B. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0851-8 · Cited by 182
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking
Ma, C., Huang, J.-B., Yang, X., & Yang, M.-H. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1076-4 · Cited by 156
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Xu, L., Yang, J., Zheng, X., Li, K., Sun, X. W., et al. (2025) · neural information processing systems · doi:10.52202/085713-4899 · Cited by 148
Token Merging: Your ViT But Faster
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2022) · arXiv · doi:10.48550/arxiv.2210.09461 · Cited by 140
Temporal Aggregate Representations for Long-Range Video Understanding
Şener, F., Singhania, D., & Yao, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_10 · Cited by 139
Learning Position and Target Consistency for Memory-based Video Object Segmentation
Hu, L., Zhang, P., Zhang, B., Pan, P., Xu, Y., & Jin, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00413 · Cited by 132
Towards Long-Form Video Understanding
Wu, C.-Y., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00192 · Cited by 131
Hierarchical Memory Matching Network for Video Object Segmentation
Seong, H., Oh, S. W., Lee, J., Lee, S., Lee, S., & Kim, E. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01265 · Cited by 129
MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptions
Soldan, M., Pardo, A., Alcázar, J. L., Heilbron, F. C., Zhao, C., Giancola, S., & Ghanem, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00497 · Cited by 94
Multiple-Target Tracking: A Role for Working Memory?
Allen, R., McGeorge, P., Pearson, D., & Milne, A. B. (2005) · Quarterly Journal of Experimental Psychology · doi:10.1080/02724980543000097 · Cited by 89
Zero-Shot Video Question Answering via Frozen Bidirectional Language ModelsPreprint
Yang, A., Miech, A., Šivic, J., Ivan Laptev, A., & Schmid, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.08155 · Cited by 87
Title Generation for User Generated Videos
Zeng, K.-H., Chen, T.-H., Niebles, J. C., & Sun, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_38 · Cited by 79
EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding
Mangalam, K., Akshulakov, R., & Malik, J. (2023) · neural information processing systems · doi:10.52202/075280-2004 · Cited by 77
MultiModal-GPT: a vision and language model for dialogue with humans
Gong, T., Lyu, C., Zhang, S., Wang, Y., Zheng, M., Zhao, Q., Liu, K., Zhang, W., Luo, P., & Chen, K. (2023) · arXiv · doi:10.48550/arxiv.2305.04790 · Cited by 70
Generic Event Boundary Detection: A Benchmark for Event Segmentation
Shou, M. Z., Lei, S. W., Wang, W., Ghadiyaram, D., & Feiszli, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00797 · Cited by 65
ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding SystemPreprint
Wang, J., Chen, D., Luo, C., Dai, X., Yuan, L., Wu, Z., & Jiang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.14407 · Cited by 63
Memory-and-Anticipation Transformer for Online Action Understanding
Wang, J., Chen, G., Huang, Y., Wang, L., & Lu, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01271 · Cited by 62
Hierarchical Video-Moment Retrieval and Step-Captioning
Zala, A., Cho, J., Kottur, S., Chen, X., Oğuz, B., Mehdad, Y., & Bansal, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02208 · Cited by 57
Notebaard, J. (2024) · doi:10.33540/2168 · Cited by 57
Generating Descriptions with Grounded and Co-referenced People
Rohrbach, A., Rohrbach, M., Tang, S., Oh, S. J., & Schiele, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.447 · Cited by 53
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 48
Self-citationAwad, G., Butt, A. A., Fiscus, J. G., Joy, D., Delgado, A., Mcclinton, W., Michel, M., Smeaton, A. F., Graham, Y., Kraaij, W., et al. (2017) · HAL · Cited by 42
(2011) · SpringerReference · doi:10.1007/springerreference_9081 · Cited by 38
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration
Lyu, C., Wu, M., Wang, L., Huang, X., Liu, B., Du, Z., Shi, S., & Tu, Z. (2023) · arXiv · doi:10.48550/arxiv.2306.09093 · Cited by 38
Memory Network With Pixel-Level Spatio-Temporal Learning for Visual Object Tracking
Zhou, Z., Zhou, X., Chen, Z., Guo, P., Liu, Q., & Zhang, W. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3272319 · Cited by 27
Awad, G., Butt, A. A., Curtis, K., Lee, Y., Fiscus, J., Godil, A., Delgado, A., Zhang, J., Godard, E., Diduch, L. L., et al. (2019) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 26
Awad, G., Butt, A. A., Curtis, K., Fiscus, J., Godil, A., Lee, Y., Delgado, A., Zhang, J., Godard, E., Chocot, B., et al. (2021) · arXiv · doi:10.48550/arxiv.2104.13473 · Cited by 18
UMOTMA: Underwater multiple object tracking with memory aggregation
Hao, Z., Qiu, J., Zhang, H., Ren, G., & Liu, C. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.1071618 · Cited by 17
MAVOT: Memory-Augmented Video Object TrackingPreprint
Liu, B., Wang, Y., Tai, Y., & Tang, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09414 · Cited by 8
Multi-Object Tracking with Spatial-Temporal Correlation Memory Networks
Xin, M., Sun, W., Li, K., & Hui, G. (2022) · 2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA) · doi:10.1109/cvidliccea56201.2022.9825193 · Cited by 2
Research works printed but not linked: 10
- 2023, Meet claude
- Awad, 2018, Trecvid 2018: Benchmarking video activity detection, video captioning and matching, video storytelling linking and video search
- 10.1109/ICCV48922.2021.00175 · DOI · Paper page
- Chiang, 2023, Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
- Driess, 2023, Palme: An embodied multimodal language model.
- 10.1007/springerreference_9081 · DOI
- 2021, Gpt3.5, 2021.
- 2023, Gpt-4 technical report.
- Su, 2023, Bert position encoding.
- Taori, 2023, Stanford alpaca: An instruction-following llama model
Cited by
Sorted by citations
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02245 · Cited by 291
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Li, Y., Wang, C., & Jia, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_19 · Cited by 403
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
Ren, S., Yao, L., Li, S., Sun, X., & Hou, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01357 · Cited by 277
VideoAgent: Long-Form Video Understanding with Large Language Model as Agent
Wang, X., Zhang, Y., Zohar, O., & Yeung, S. Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_4 · Cited by 196
Self-citation
Metadata
- Authors
- Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., Lu, Y., Hwang, J.-N., & Wang, G.
- Affiliations
- Zhejiang University (CN)
- University of Washington (US)
- Donghua University (CN)
- Microsoft Research Asia (China) (CN)
- Hong Kong University of Science and Technology (GZ)
- ORCID
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 18221-18232
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402702917
- Cited by
- 176
- References
- 76
- Citations by year
- 2026: 82 · 2025: 60 · 2024: 32
- FWCI
- 35.92
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Key R&D Program of China (2022ZD0162000)
- National Natural Science Foundation of China (62106219,62206046)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Keywords
- long video understanding, video foundation models, large language models, temporal relations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
