The DOI matches its Crossref record; 68 of the 69 references it cites were found in a registry or an index.
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Open accessAbstract
Learning text-video embeddings usually requires a dataset of video clips with manually provided captions. However, such datasets are expensive and time consuming to create and therefore difficult to obtain on a large scale. In this work, we propose instead to learn such embeddings from video data with readily available natural language annotations in the form of automatically transcribed narrations. The contributions of this work are three-fold. First, we introduce HowTo100M: a large-scale dataset of 136 million video clips sourced from 1.22M narrated instructional web videos depicting humans performing and describing over 23k different visual tasks. Our data collection procedure is fast, scalable and does not require any additional manual annotation. Second, we demonstrate that a text-video embedding trained on this data leads to state-of-the-art results for text-to-video retrieval and action localization on instructional video datasets such as YouCook2 or CrossTask. Finally, we show that this embedding transfers well to other domains: fine-tuning on generic Youtube videos (MSR-VTT dataset) and movies (LSMDC dataset) outperforms models trained on these datasets alone. Our dataset, code and models are publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The beginning and end of videos, especially tutorial videos (Miech et al., 2019), often consist of an intro, respectively an outro. These sections are not as related to the content of the videos and contain fewer movements and actions.”
From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Miech et al., 2019
Experimental — accuracy not yet measuredIn the citing paper
“More modalities.In addition to the general modalities of vision, text, and audio, further diverse modalities are emerging, e.g., Pano-AVQA [144] -the first large-scale spatial and audio-visual question answering dataset on 360 • videos, YouTube-360 (YT-360) [145] (360 • videos), AIST++ [146] (a new multimodal dataset of 3D dance motion and music), Artemis [147] (affective language for visual arts)”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [144]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
Hara, K., Kataoka, H., & Satoh, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00685 · Cited by 2,346
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,888
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869
Flagged first, then by citations
1 of the 69 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- radford, 2018, Improving Language Understandingby Generative Pre-Training, Preprint
- 2019, Project webpage
- radford, 2019, Language models are unsupervised multitask learners, Preprint
- damen, 2018, Scaling egocentric vision: The epic-kitchens dataset, ECCV
Cited by
Sorted by citations
Items 501–600 of 989
Egocentric Biochemical Video-and-Language Dataset
Nishimura, T., Sakoda, K., Hashimoto, A., Ushiku, Y., Tanaka, N., Ono, F., Kameko, H., & Mori, S. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00348 · Cited by 8
Video-Text Pre-training with Learned RegionsPreprint
Yan, R., Shou, M. Z., Ge, Y., Wang, A. J., Lin, X., Cai, G., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01194 · Cited by 8
Watch Less and Uncover More: Could Navigation Tools Help Users Search and Explore Videos?Preprint
Pérez‐Ortiz, M., Bulathwela, S., Dormann, C., Verma, M., Kreitmayer, S., Noss, R., Shawe-Taylor, J. S., Rogers, Y., & Yılmaz, E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.03408 · Cited by 8
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v2 · Cited by 11
Action knowledge for video captioning with graph neural networks
Hendria, W. F., Velda, V., Putra, B. H. H., Adzaka, F., & Jeong, C. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.03.006 · Cited by 8
GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations
Ilaslan, M. F., Song, C., Chen, J., Gao, D., Lei, W., Xu, Q., Lim, J., & Shou, M. Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.648 · Cited by 8
Analyzing and Predicting Consumer Response to Short Videos in E-Commerce
Guo, Y., Ban, C., Yang, J., Goh, K. Y., Liu, X. J., Peng, X., & Li, X. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3690393 · Cited by 12
VidLA: Video-Language Alignment at Scale
Rizve, M. N., Fei, F., Unnikrishnan, J., Tran, S., Yao, B., Zeng, B., Shah, M., & Chilimbi, T. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01332 · Cited by 7
VideoA11y: Method and Dataset for Accessible Video Description
Li, C., Padmanabhuni, S., Cheema, M. S., Seifi, H., & Fazli, P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714096 · Cited by 7
HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generation
Liu, K., Liu, Q., Liu, X., Li, J., Zhang, Y., Luo, J., He, X., & Liu, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02235 · Cited by 7
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
Chen, K., Gou, Y., Huang, R., Liu, Z., Tan, D., Xu, J., Wang, C., Zhu, Y., Zeng, Y., Yang, K., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00513 · Cited by 49
Basket: A Large-Scale Video Dataset for Fine-Grained Skill Estimation
Pan, Y., Zhang, C., & Bertasius, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02696 · Cited by 7
A Comprehensive Survey on Text-to-Video Generation
Xie, F., Zeng, D., Shen, Q., & Tang, B. (2025) · Chinese Journal of Electronics · doi:10.23919/cje.2024.00.151 · Cited by 12
Taming Data and Transformers for Audio Generation
Haji-Ali, M., Menapace, W., Siarohin, A., Balakrishnan, G., & Ordóñez, V. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02632-y · Cited by 31
Recent Advances in Video Action Recognition with 3D Convolutions
Hara, K. (2020) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.2020imp0012 · Cited by 9
Pal, R., Sekh, A. A., Dogra, D. P., Kar, S., Roy, P. P., & Prasad, D. K. (2021) · ACM Computing Surveys · doi:10.1145/3459089 · Cited by 7
TEVL: Trilinear Encoder for Video-language Representation Learning
Man, X., Shao, J., Chen, F., Zhang, M., & Shen, H. T. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3585388 · Cited by 8
A comprehensive survey on deep-learning-based visual captioning
Xin, B., Xu, N., Zhai, Y., Zhang, T., Lu, Z., Liu, J., Nie, W., Li, X., & Liu, A.-A. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01175-x · Cited by 8
ConGMC: Consistency-Guided Multimodal Clustering via Mutual Information Maximin
Mao, Y., Yan, X., Liu, J., & Ye, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3330093 · Cited by 7
Stitching Segments and Sentences towards Generalization in Video-Text Pre-training
Ma, F., Jin, X., Wang, H., Huang, J., Zhu, L., & Yang, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28202 · Cited by 7
Automated Sperm Assessment Framework and Neural Network Specialized for Sperm Video Recognition
Fujii, T., Nakagawa, H., Takeshima, T., Yumura, Y., & Hamagami, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00750 · Cited by 9
See, Predict, Plan: Diffusion for Procedure Planning in Robotic Surgical Videos
Zhao, Z., Fang, F., Yang, X., Xu, Q., Guan, C., & Zhou, S. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72089-5_52 · Cited by 7
VideoMix: Aggregating How-To Videos for Task-Oriented Learning
Yang, S., Truong, A., Kim, J., & Li, D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712144 · Cited by 7
MultiModal Language Modelling on Knowledge Graphs for Deep Video Understanding
Anand, V., Ramesh, R., Jin, B., Wang, Z., Lei, X., & Lin, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3479220 · Cited by 7
Know your sensors — a modality study for surgical action classification
Bastian, L., Czempiel, T., Heiliger, C., Karcz, K. W., Eck, U., Busam, B., & Navab, N. (2022) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2022.2152377 · Cited by 7
C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval
Rouditchenko, A., Chuang, Y.-S., Shvetsova, N., Thomas, S., Feris, R., Kingsbury, B. E. D., Karlinsky, L., Harwath, D. F., Kuehne, H., & Glass, J. R. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094821 · Cited by 7
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts
Engin, D., & Avrithis, Y. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00298 · Cited by 10
Ko, D., Lee, J. S., Choi, M., Chu, J., Park, J., & Kim, H. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00288 · Cited by 8
Mutual modality learning for video action classification
Komkov, S. A., Dzabraev, M., & Petiushko, A. (2023) · Computer Optics · doi:10.18287/2412-6179-co-1277 · Cited by 8
Semantic Role Aware Correlation Transformer For Text To Video Retrieval
Satar, B., Zhu, H., Bresson, X., & Lim, J. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506267 · Cited by 9
Expert Training: Task Hardness Aware Meta-Learning for Few-Shot ClassificationPreprint
Zhou, Y., Wang, Y., Cai, J., Yu, Z., Hu, Q., & Wang, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06240 · Cited by 10
Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learningPreprint
Cabannes, V., Pillaud‐Vivien, L., Bach, F., & Rudi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.04324 · Cited by 7
Support-Set Based Cross-Supervision for Video GroundingPreprint
Ding, X., Wang, N., Zhang, S., Cheng, D., Li, X., Huang, Z., Tang, M., & Gao, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10576 · Cited by 7
A CLIP-Enhanced Method for Video-Language UnderstandingPreprint
Li, G., He, F., & Feng, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07137 · Cited by 7
Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video RetrievalPreprint
Yang, Y., Kim, J., Panagopoulou, A., Yatskar, M., & Callison-Burch, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09276 · Cited by 12
Category-Specific Prompts for Animal Action Recognition with Pretrained Vision-Language Models
Jing, Y., Wang, C., Zhang, R., Liang, K., & Ma, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612551 · Cited by 6
Enhancing Text-Video Retrieval Performance With Low-Salient but Discriminative Objects
Zheng, Y., Huang, B., Chen, Z., & Yu, D. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3527369 · Cited by 6
Multi-Grained Alignment with Knowledge Distillation for Partially Relevant Video Retrieval
Zhang, Q., Yang, C., Jiang, B., & Zhang, B. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3716388 · Cited by 6
SHE-Net: Syntax-Hierarchy-Enhanced Text-Video Retrieval
Yu, X., Jiang, C., Dong, X., Gan, T., Yang, M., & Guo, Q. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3543840 · Cited by 6
Animal-CLIP: A Dual-Prompt Enhanced Vision-Language Model for Animal Action Recognition
Jing, Y., Liang, K., Zhang, R., Sun, H., Li, Y., He, Z., & Ma, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02408-4 · Cited by 8
IVCR-200K: A Large-Scale Multi-turn Dialogue Benchmark for Interactive Video Corpus Retrieval
Han, N., YaWen, Z., Long, S., Li, C., Yang, S., Tan, D., Dong, J., & Chen, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730015 · Cited by 7
Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning
Li, J. W., Wang, J., Tan, C., Lian, N., Chen, L., Wang, Y., Zhang, M., Xia, S., & Chen, B. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02142 · Cited by 6
VideoCon: Robust Video-Language Alignment via Contrast Captions
Bansal, H., Bitton, Y., Szpektor, I., Chang, K.-W., & Grover, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01321 · Cited by 13
PARIS3D: Reasoning-Based 3D Part Segmentation Using Large Multimodal Model
Kareem, A., Lahoud, J., & Cholakkal, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73220-1_27 · Cited by 6
Text-driven affordance learning from egocentric vision
Yoshida, T., Kurita, S., Nishimura, T., & Mori, S. (2025) · Advanced Robotics · doi:10.1080/01691864.2025.2535676 · Cited by 10
Research Progress on Vision–Language Multimodal Pretraining Model Technology
Wang, H., Huang, R., & Zhang, J. (2022) · Electronics · doi:10.3390/electronics11213556 · Cited by 6
Relational Future Captioning Model for Explaining Likely Collisions in Daily Tasks
Kambara, M., & Sugiura, K. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897231 · Cited by 6
Learning Object Manipulation Skills from Video via Approximate Differentiable Physics
Petrík, V., Qureshi, M. N., Šivic, J., & Tapaswi, M. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982084 · Cited by 6
Self-citationExploring TV Seriality and Television Studies through Data-Driven Approaches
Esposti, M. D., & Pescatore, G. (2023) · doi:10.21428/93b7ef64.ec022085 · Cited by 7
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training
Zhong, W., Zheng, M. S., Tang, D., Luo, X., Gong, H., Feng, X., & Qin, B. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25483 · Cited by 6
Li, M., Shi, X., Leng, H., Zhou, W. G., Zheng, H.-T., & Zhang, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25222 · Cited by 7
MemBridge: Video-Language Pre-Training With Memory-Augmented Inter-Modality Bridge
Yang, J., Li, X., Zheng, M., Wang, Z., Zhu, Y., Guo, X., Yuan, Y., Chai, Z., & Jiang, S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3283916 · Cited by 6
Learning Action Changes by Measuring Verb-Adverb Textual Relationships
Moltisanti, D., Keller, F., Bilen, H., & Sevilla-Lara, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02213 · Cited by 6
Attentive Snippet Prompting for Video Retrieval
Chen, S., Xu, Q., Ma, Y., Qiao, Y., & Wang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3321503 · Cited by 9
TeViS: Translating Text Synopses to Video Storyboards
Gu, X., Sun, Y., Ni, F., Chen, S., Wang, X., Song, R., Li, B., & Cao, X. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612417 · Cited by 12
Himakunthala, V., Ouyang, A., Rose, D., He, R., Mei, A., Lu, Y., Sonar, C., Saxon, M., & Wang, W. Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.15 · Cited by 19
Exploring Temporal Concurrency for Video-Language Representation Learning
Zhang, H., Liu, D., Lv, Z., Su, B., & Tao, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01427 · Cited by 6
YTCommentQA: Video Question Answerability in Instructional Videos
Yang, S., Park, S., Jang, Y., & Lee, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29906 · Cited by 11
Detours for Navigating Instructional Videos
Ashutosh, K., Xue, Z., Nagarajan, T., & Grauman, K. L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01779 · Cited by 7
Visual Grounding in Video for Unsupervised Word Translation
Sigurdsson, G. A., Alayrac, J.-B., Nematzadeh, A., Smaira, L., Malinowski, M., Carreira, J., Blunsom, P., & Zisserman, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01086 · Cited by 6
Self-citationScreencast Tutorial Video Understanding
Li, K., Chen, F., Wang, Z., Kim, S., Jin, H., & Fu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01254 · Cited by 6
What Matters for Ad-hoc Video Search? A Large-scale Evaluation on TRECVID
Chen, A., Hu, F., Wang, Z., Zhou, F., & Li, X. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00262 · Cited by 6
SemEval-2022 Task 9: R2VQ – Competence-based Multimodal Question Answering
Tu, J., Holderness, E., Maru, M., Conia, S., Rim, K., Lynch, K., Brutti, R. A., Roberto Navigli, A., & Pustejovsky, J. D. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.176 · Cited by 6
Tencent-MVSE: A Large-Scale Benchmark Dataset for Multi-Modal Video Similarity Evaluation
Zeng, Z., Luo, Y., Liu, Z., Rao, F., Li, D., Guo, W., & Wen, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00314 · Cited by 7
CNVid-3.5M: Build, Filter, and Pre-Train the Large-Scale Public Chinese Video-Text Dataset
Gan, T., Wang, Q., Dong, X., Ren, X., Nie, L., & Guo, Q. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01423 · Cited by 6
Multi-modal Transformer for Video RetrievalPreprint
Gabeur, V., Sun, C., Alahari, K., & Schmid, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10639 · Cited by 11
Disambiguation of weak supervision with exponential convergence ratesPreprint
Cabannes, V., Bach, F., & Rudi, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02789 · Cited by 6
CUPID: Adaptive Curation of Pre-training Data for Video-and-Language Representation LearningPreprint
Zhou, L., Liu, J., Yu, C., Gan, Z., & Zhang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00285 · Cited by 8
VLM: Task-agnostic Video-Language Model Pre-training for Video UnderstandingPreprint
Xu, H., Ghosh, G., Huang, P.-Y., Arora, P., Aminzadeh, M., Feichtenhofer, C., Metze, F., & Zettlemoyer, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09996 · Cited by 21
M5Product: A Multi-modal Pretraining Benchmark for E-commercial Product Downstream TasksPreprint
Dong, X. M., Zhan, X., Wu, Y., Wei, Y., & Liang, X. (2021) · arXiv (Cornell University) · doi:10.13140/rg.2.2.35620.81281 · Cited by 6
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
NVIDIA, :, Bjorck, J., Castañeda, F., Cherniadev, N., Da, X., Ding, R., Fan, L. "., Fang, Y., Fox, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.14734 · Cited by 10
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R., Mojtaba, Komeili, Muckley, M., Rizvi, A., Roberts, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.09985 · Cited by 8
“The overall dataset includes ego-centric videos from the Something-Something v2 dataset (SSv2) introduced in Goyal et al. (2017), exo-centric action videos from the Kinetics 400, 600, and 700 datasets (Kay et al., 2017; Carreira et al., 2018, 2019), YouTube tutorial videos from HowTo100M (Miech et al., 2019), and gene…”
Chen, Y., Li, K., Bao, W., Patel, D., Kong, Y., Min, M. R., & Metaxas, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73007-8_12 · Cited by 7
Simplifying Traffic Anomaly Detection with Video Foundation Models
Orlova, S., Kerssies, T., Englert, B. B., & Dubbelman, G. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00094 · Cited by 5
A video dataset of a wooden box assembly process
Zhang, J., Byvshev, P., & Xiao, Y. (2020) · Workshop on Data: Acquisition To Analysis (DATA) · doi:10.1145/3419016.3431492 · Cited by 5
Simultaneously Training and Compressing Vision-and-Language Pre-Training Model
Qi, Q., Zhang, A., Liao, Y., Sun, W., Wang, Y., Li, X., & Liu, S. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3233258 · Cited by 6
Feature distillation from vision-language model for semisupervised action classification
ÇELİK, A., Küçükmanіşa, A., & Urhan, O. (2023) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.55730/1300-0632.4038 · Cited by 5
Language-aware Visual Semantic Distillation for Video Question Answering
Zou, B., Chao, Y., Qiao, Y., Quan, C., & Zhao, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02560 · Cited by 9
Exploiting Unlabeled Videos for Video-Text Retrieval via Pseudo-Supervised Learning
Lu, Y., Quan, R., Zhu, L., & Yang, Y. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3514352 · Cited by 5
LORD: Large Models Based Opposite Reward Design for Autonomous Driving
Ye, X., Tao, F., Mallik, A., Yaman, B., & Ren, L. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00496 · Cited by 8
Learning Multiple Object States from Actions via Large Language Models
Tateno, M., Yagi, T., Furuta, R., & Sato, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00925 · Cited by 9
FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation
Yagi, T., Ohashi, M., Huang, Y., Furuta, R., Adachi, S., Mitsuyama, T., & Sato, Y. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02523-2 · Cited by 5
WhyAct: Identifying Action Reasons in Lifestyle Vlogs
Ignat, O., Castro, S., Miao, H., Li, W., & Mihalcea, R. F. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.392 · Cited by 5
Generalizable Multi-linear Attention Network
Jin, T., & Zhao, Z. (2021) · neural information processing systems · Cited by 5
Video Language Co-Attention with Multimodal Fast-Learning Feature Fusion for VideoQA
Abdessaied, A., Sood, E., & Bulling, A. (2022) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2022.repl4nlp-1.15 · Cited by 5
Cascaded MPN: Cascaded Moment Proposal Network for Video Corpus Moment Retrieval
Yoon, S., Kim, D., Kim, J., & Yoo, C. D. (2022) · IEEE Access · doi:10.1109/access.2022.3183106 · Cited by 5
Multi-Level Cross-Modal Semantic Alignment Network for Video–Text Retrieval
Nian, F., Ding, L., Hu, Y., & Gu, Y. (2022) · Mathematics · doi:10.3390/math10183346 · Cited by 5
From Within to Between: Knowledge Distillation for Cross Modality Retrieval
Tran, V., Balasubramanian, N., & Hoai, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_36 · Cited by 5
Temporal Natural Language Inference: Evidence-Based Evaluation of Temporal Text Validity
Hosokawa, T., Jatowt, A., & Sugiyama, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28244-7_28 · Cited by 5
Video question answering supported by a multi-task learning objective
Falcon, A., Serra, G., & Lanz, O. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14333-0 · Cited by 5
Non-Sequential Graph Script Induction via Multimedia Grounding
Zhou, Y., Li, S., Li, M., Lin, X., Chang, S.-F., Bansal, M., & Ji, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.303 · Cited by 5
GHAN: Graph-Based Hierarchical Aggregation Network for Text-Video Retrieval
Yu, Y., Hu, B., & Li, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.374 · Cited by 5
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant
Lei, W., Gao, D., Wang, Y., Mao, D., Liang, Z., Ran, L., & Shou, M. Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.24 · Cited by 5
PWS-DVC: Enhancing Weakly Supervised Dense Video Captioning With Pretraining Approach
Choi, W., Chen, J., & Yoon, J. (2023) · IEEE Access · doi:10.1109/access.2023.3331756 · Cited by 5
M2C: Towards Automatic Multimodal Manga Complement
Guo, H., Wang, B., Bai, J., Liu, J., Yang, J., & Li, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.661 · Cited by 6
Chinese Title Generation for Short Videos: Dataset, Metric and Algorithm
Zhang, Z., Ma, Z., Yuan, C., Chen, Y., Wang, P., Qi, Z., Hao, C., Li, B., Shan, Y., Hu, W., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3365739 · Cited by 6
Video Event Extraction with Multi-View Interaction Knowledge Distillation
Wei, K., Du, R., Jin, L., Liu, J., Yin, J., Zhang, L., Liu, J., Liu, N., Zhang, J., & Guo, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29891 · Cited by 5
Learning with Noisy Correspondence
Huang, Z., Hu, P., Niu, G., Xiao, X., Lv, J., & Peng, X. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02064-0 · Cited by 5
Exploiting Instance-level Relationships in Weakly Supervised Text-to-Video Retrieval
Yin, S., Zhao, S., Wang, H., Xu, T., & Chen, E. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3663571 · Cited by 5
Cross-Lingual Cross-Modal Retrieval With Noise-Robust Fine-Tuning
Cai, R., Dong, J., Liang, T., Liang, Y., Wang, Y., Yang, X., Wang, X., & Wang, M. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3400060 · Cited by 8
Metadata
- Authors
- Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2630-2640
- OpenAlex
- W2948859046
- Cited by
- 986
- References
- 69
- Citations by year
- 2026: 77 · 2025: 138 · 2024: 170 · 2023: 213 · 2022: 161 · 2021: 155 · 2020: 65 · 2019: 7
- FWCI
- 31.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- European Regional Development Fund (02.1.01/0.0/0.0, CZ.02, CZ.02.1.01, CZ.02.1.01/0.0, CZ.02.1.01/0.0/0.0)
- FP7 Ideas: European Research Council (336845)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video-text retrieval, action localization, domain transfer, large-scale video datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
