The DOI matches its Crossref record; 68 of the 69 references it cites were found in a registry or an index.
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Open accessAbstract
Learning text-video embeddings usually requires a dataset of video clips with manually provided captions. However, such datasets are expensive and time consuming to create and therefore difficult to obtain on a large scale. In this work, we propose instead to learn such embeddings from video data with readily available natural language annotations in the form of automatically transcribed narrations. The contributions of this work are three-fold. First, we introduce HowTo100M: a large-scale dataset of 136 million video clips sourced from 1.22M narrated instructional web videos depicting humans performing and describing over 23k different visual tasks. Our data collection procedure is fast, scalable and does not require any additional manual annotation. Second, we demonstrate that a text-video embedding trained on this data leads to state-of-the-art results for text-to-video retrieval and action localization on instructional video datasets such as YouCook2 or CrossTask. Finally, we show that this embedding transfers well to other domains: fine-tuning on generic Youtube videos (MSR-VTT dataset) and movies (LSMDC dataset) outperforms models trained on these datasets alone. Our dataset, code and models are publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The beginning and end of videos, especially tutorial videos (Miech et al., 2019), often consist of an intro, respectively an outro. These sections are not as related to the content of the videos and contain fewer movements and actions.”
From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Miech et al., 2019
Experimental — accuracy not yet measuredIn the citing paper
“More modalities.In addition to the general modalities of vision, text, and audio, further diverse modalities are emerging, e.g., Pano-AVQA [144] -the first large-scale spatial and audio-visual question answering dataset on 360 • videos, YouTube-360 (YT-360) [145] (360 • videos), AIST++ [146] (a new multimodal dataset of 3D dance motion and music), Artemis [147] (affective language for visual arts)”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [144]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
Hara, K., Kataoka, H., & Satoh, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00685 · Cited by 2,346
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,888
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869
Research works printed but not linked: 4
- radford, 2018, Improving Language Understandingby Generative Pre-Training, Preprint
- 2019, Project webpage
- radford, 2019, Language models are unsupervised multitask learners, Preprint
- damen, 2018, Scaling egocentric vision: The epic-kitchens dataset, ECCV
Cited by
Sorted by citations
Items 601–700 of 990
Cross-Lingual Cross-Modal Retrieval With Noise-Robust Fine-Tuning
Cai, R., Dong, J., Liang, T., Liang, Y., Wang, Y., Yang, X., Wang, X., & Wang, M. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3400060 · Cited by 8
Optimizing Video Selection LIMIT Queries with Commonsense Knowledge
He, W., Sabek, I., Lou, Y., & Cafarella, M. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3654621.3654639 · Cited by 5
Video Recognition in Portrait Mode
Han, M., Yang, L., Jin, X., Feng, J., Chang, X., & Wang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02062 · Cited by 5
Exploring the Role of Audio in Video Captioning
Shen, Y., Yang, L., Wen, L., Yu, H., Elhamifar, E., & Wang, H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00214 · Cited by 5
SnapCap: efficient snapshot compressive scene captioning
Sun, J., Su, Y., Zhang, H., Cheng, Z., Zeng, Z., Wang, Z., Qu, C., Chen, B., & Yuan, X. (2025) · Advanced imaging. · doi:10.3788/ai.2025.10021 · Cited by 5
The Journey of Action Recognition
Xi, D., & Wang, L. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717746 · Cited by 6
Grounded Sequence to Sequence Transduction
Specia, L., Barrault, L., Çağlayan, O., Duarte, A., Elliott, D., Gella, S., Holzenberger, N., Lala, C., Lee, S. J., Libovický, J., et al. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2998415 · Cited by 5
Relevance-based Margin for Contrastively-trained Video Retrieval Models
Falcon, A., Sudhakaran, S., Serra, G., Escalera, S., & Lanz, O. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531395 · Cited by 5
Video Sampled Frame Category Aggregation and Consistent Representation for Cross-Modal Retrieval
Jin, M., Zhang, H., Zhu, L., Sun, J., & Liu, L. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3204623 · Cited by 5
BioVL2: An Egocentric Biochemical Video-and-Language Dataset
Nishimura, T., Sakoda, K., Ushiku, A., Hashimoto, A., Okuda, N., Ono, F., Kameko, H., & Mori, S. (2022) · Journal of Natural Language Processing · doi:10.5715/jnlp.29.1106 · Cited by 5
Wang, H., Li, H., Liu, W., & Gu, X. D. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104785 · Cited by 5
Grafting Pre-trained Models for Multimodal Headline Generation
Qiao, L., Wu, C., Liu, Y., Peng, H., Yin, D., & Ren, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.25 · Cited by 5
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language ModelsPreprint
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.08849 · Cited by 5
Film Trailer Generation via Task Decomposition.Preprint
Papalampidi, P., Keller, F., & Lapata, M. (2021) · arXiv (Cornell University) · Cited by 5
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text UnderstandingPreprint
Xu, H., Ghosh, G., Huang, P.-Y., Okhonko, D., Aghajanyan, A., Metze, F., Zettlemoyer, L., & Feichtenhofer, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.14084 · Cited by 221
Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrieval
Du, Y., Liu, Y., & Jin, Q. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680731 · Cited by 4
Vinci: A Real-time Smart Assistant Based on Egocentric Vision-language Model for Portable Devices
Huang, Y., Xu, J., Pei, B., Yang, L., Zhang, M., He, Y., Guo, C., Chen, X., Wang, Y., Nie, Z., et al. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749513 · Cited by 37
Wu, J., Ngo, C., & Chan, W. K. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658052 · Cited by 4
Du, J.-R., Lin, K.-Y., Meng, J., & Zheng, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78444-6_17 · Cited by 4
Level-wise aligned dual networks for text–video retrieval
Lin, Q., Cao, W., & He, Z. (2022) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-022-00887-y · Cited by 7
Chan, D. M., Myers, A., Vijayanarasimhan, S., Ross, D. A., Seybold, B., & Canny, J. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00520 · Cited by 6
Simultaneous Video Retrieval and Alignment
Jo, W., Lim, G., Hwang, Y., Lee, G., Kim, J., Yun, J., Jung, J., & Choi, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3259733 · Cited by 4
Foundation Models for Speech, Images, Videos, and Control
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_7 · Cited by 6
Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval
Yakovlev, K., Polyakov, G., Alimova, I., Podolskiy, A., Bout, A., Nikolenko, S., & Piontkovskaya, I. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592064 · Cited by 4
Beyond Grounding: Extracting Fine-Grained Event Hierarchies across Modalities
Ayyubi, H., Thomas, C., Chum, L., Lokesh, R., Chen, L., Niu, Y., Lin, X., Feng, X., Koo, J., Ray, S., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29718 · Cited by 4
From Viewers to Teachers: Child-Led Teaching Strategies and Family Participation in YouTube How-Tos
Cai, Z., & Wei, S. (2024) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3628516.3655821 · Cited by 4
Lee, J.-H., & Kang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01299 · Cited by 4
ICSVR: Investigating Compositional and Syntactic Understanding in Video Retrieval Models
Madasu, A., & Lal, V. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00180 · Cited by 5
Unsupervised Video Moment Retrieval with Knowledge-Based Pseudo-Supervision Construction
Wang, G., Wu, X., Tu, X., Liu, Z., & Yan, J. (2024) · ACM Transactions on Information Systems · doi:10.1145/3701229 · Cited by 5
Multi-sentence Grounding for Long-Term Instructional Video
Li, Z., Chen, Q., Han, T., Zhang, Y., Wang, Y., & Xie, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_12 · Cited by 12
Day2Dark: Pseudo-Supervised Activity Recognition Beyond Silent Daylight
Zhang, Y., Doughty, H., & Snoek, C. G. M. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02273-7 · Cited by 4
Audio-Language Datasets of Scenes and Events: A Survey
Wijngaard, G., Formisano, E., Esposito, M., & Dumontier, M. (2025) · IEEE Access · doi:10.1109/access.2025.3534621 · Cited by 7
Ohkawa, T., Yagi, T., Nishimura, T., Furuta, R., Hashimoto, A., Ushiku, Y., & Sato, Y. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00807 · Cited by 4
Enhancing Text-to-Video Retrieval Using Clip Based Deep Learning Approach
Adil, M., Akhtar, N., Khan, S., & Shoaib, H. (2025) · IEEE International Conference on Communication Systems and Network Technologies (CSNT) · doi:10.1109/csnt64827.2025.10968134 · Cited by 4
Majumder, S., Nagarajan, T., Al-Halah, Z., Pradhan, R., & Grauman, K. L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02702 · Cited by 9
Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy
Pattnayak, P., Patel, H. L., Kumar, B., Agarwal, A., Banerjee, I., Panda, S., & Kumar, T. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-14041-8_4 · Cited by 30
Multi-grained encoding and joint embedding space fusion for video and text cross-modal retrieval
Cui, X., Xiao, J., Cao, Y., & Zhu, J. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13048-y · Cited by 4
Learning Linguistic Association Towards Efficient Text-Video Retrieval
Fang, S., Wang, S., Zhuo, J., Han, X., & Huang, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_15 · Cited by 4
Recipe2Video: Synthesizing Personalized Videos from Recipe Texts
Udhayanan, P., Bv, S., Laturia, P., Chauhan, D., Khandelwal, D., Petrangeli, S., & Srinivasan, B. V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00230 · Cited by 4
CMMT: Cross-Modal Meta-Transformer for Video-Text Retrieval
Gao, Y., & Lu, Z. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592238 · Cited by 4
Mask to Reconstruct: Cooperative Semantics Completion for Video-text Retrieval
Fang, H., Yang, Z., Zang, X., Ban, C., He, Z., Sun, H. S., & Zhou, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611756 · Cited by 4
HiVLP: Hierarchical Interactive Video-Language Pre-Training
Shao, B., Liu, J., Pei, R., Xu, S., Dai, P., Lu, J., Li, W., & Yan, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01265 · Cited by 4
Chen, X., Yang, J., Chen, S., Wang, L., Jiang, M. X., & Zhao, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357631 · Cited by 4
Liu, Y., Shen, T., Zhang, D., Sun, Q., Li, S., & Zhou, G. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658094 · Cited by 4
Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube
Hessel, J., Zhu, Z., Pang, B., & Soricut, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.709 · Cited by 4
Cascaded Multilingual Audio-Visual Learning from Videos
Rouditchenko, A., Boggust, A., Harwath, D. F., Thomas, S., Kuehne, H., Chen, B., Panda, R., Feris, R., Kingsbury, B. E. D., Picheny, M., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1352 · Cited by 4
Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset
Palmer, I., Rouditchenko, A., Barbu, A., Katz, B., & Glass, J. R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-245 · Cited by 4
Universal Prototype Transport for Zero-Shot Action Recognition and Localization
Mettes, P. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01846-2 · Cited by 4
Space-Time Crop & Attend: Improving Cross-Modal Video Representation LearningPreprint
Patrick, M., Asano, Y. M., Huang, B., Misra, I., Metze, F., Henriques, J., & Vedaldi, A. (2021) · arXiv (Cornell University) · Cited by 4
Unsupervised Action Segmentation for Instructional VideosPreprint
Piergiovanni, A., Angelova, A., Ryoo, M. S., & Essa, I. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03738 · Cited by 4
Visual Spatio-temporal Relation-enhanced Network for Cross-modal Text-Video RetrievalPreprint
Han, N., Chen, J., Xiao, G., YaWen, Z., Shi, C., & Chen, H. (2021) · arXiv (Cornell University) · Cited by 4
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
Ventura, L., Yang, A., Schmid, C., & Varol, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01765 · Cited by 3
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos referring to Procedural Texts
Haneji, Y., Nishimura, T., Kameko, H., Shirai, K., Yoshida, T., Kajimura, K., Yamamoto, K., Cui, T., Nishimoto, T., & Mori, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00282 · Cited by 3
Huang, L., Liu, Y., Zhou, X., You, A., Li, M., Wang, B., Zhang, Y., Pan, P., & Xu, Y. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3481541 · Cited by 3
Comment-Aware Multi-Modal Heterogeneous Pre-Training for Humor Detection in Short-Form Videos
Liu, Y., Ping, H., Zhang, D., Sun, Q., Li, S., & Zhou, G. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230438 · Cited by 3
Learning Semantics-Grounded Vocabulary Representation for Video-Text Retrieval
Shi, Y., Liu, H., Xu, H., Ma, Z., Ye, Q., Hu, A., Yan, M., Zhang, J., Huang, F., Yuan, C., et al. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612537 · Cited by 3
Scene Retrieval in Traffic Videos with Contrastive Multimodal Learning
Sadiq, T., & Omlin, C. W. P. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00153 · Cited by 3
How Does Contrastive Learning Organize Images?
Zhang, Y., Lu, Y., & Xuan, Q. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00058 · Cited by 3
Dong, X., Feng, Z., Zhou, C., Yu, X., Yang, M., & Guo, Q. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657833 · Cited by 3
Enhancing How People Learn Procedural Tasks Through How-to Videos
Yang, S. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3672539.3686711 · Cited by 3
Diffusion-Based Multimodal Video Captioning
Kainulainen, J., Guo, Z., & Laaksonen, J. T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_9 · Cited by 3
Video Editing for Video Retrieval
Zhu, B., Flanagan, K., Fragomeni, A., Wray, M., & Damen, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92591-7_15 · Cited by 3
Video Frame Enhancement based Text Semantic Fusion for Cross-modal Text-video Retrieval
Du, K., Zhang, H., Liu, L., Liu, D., & Du, H. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733460 · Cited by 3
Shen, L., Hao, T., He, T., Zhang, Y., Liu, P., Zhao, S., Han, J., & Ding, G. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3605657 · Cited by 3
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V., Bansal, S., Boote, B., et al. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02557-6 · Cited by 3
Ingredient-enriched Recipe Generation from Cooking Videos
Wu, J., Pan, L., Chen, J., & Jiang, Y. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531388 · Cited by 3
Tencent AVS: A Holistic Ads Video Dataset for Multi-Modal Scene Segmentation
Jiang, J., Li, Z., Xiong, J., Quan, R., Lu, Q., & Liu, W. (2022) · IEEE Access · doi:10.1109/access.2022.3227425 · Cited by 3
SVGraph: Learning Semantic Graphs from Instructional Videos
Schiappa, M. C., & Rawat, Y. S. (2022) · IEEE International Conference on Multimedia Big Data · doi:10.1109/bigmm55396.2022.00014 · Cited by 4
DAKRS: Domain Adaptive Knowledge-Based Retrieval System for Natural Language-Based Vehicle Retrieval
Ha, S. V., Le, H. D.-A., Nguyen, Q. Q.-V., & Chung, N. M. (2023) · IEEE Access · doi:10.1109/access.2023.3260149 · Cited by 3
Cali-NCE: Boosting Cross-modal Video Representation Learning with Calibrated Alignment
Zhao, N., Jiao, J., Xie, W., & Lin, D. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00672 · Cited by 3
A cross-modal conditional mechanism based on attention for text-video retrieval
Du, W., Jing, X., Zhu, Q., Wang, X., & Liu, X. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023889 · Cited by 3
Enhancing Ocean Scene Video Captioning with Multimodal Pre-Training and Video-Swin-Transformer
Chen, X., Zhao, M. Y., Shi, F., Zhang, M., He, Y. H., & Chen, S. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10312358 · Cited by 3
Understanding Video Scenes through Text: Insights from Text-based Video Question Answering
Jahagirdar, S., Mathew, M., Karatzas, D., & Jawahar, C. V. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00500 · Cited by 3
MMTF: Multi-Modal Temporal Fusion for Commonsense Video Question Answering
Ahmad, M., Park, G., Park, D., & Park, S. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00502 · Cited by 3
Preserving Modality Structure Improves Multi-Modal Learning
Swetha, S., Rizve, M. N., Shvetsova, N., Kuehne, H., & Shah, M. A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02010 · Cited by 4
Improving semantic video retrieval models by training with a relevance-aware online mining strategy
Falcon, A., Serra, G., & Lanz, O. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104035 · Cited by 3
Generating Illustrated Instructions
Menon, S., Misra, I., & Girdhar, R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00600 · Cited by 3
OpenStory: A Large-Scale Open-Domain Dataset for Subject-Driven Visual Storytelling
Ye, Z., Liu, J., Cao, J., Chen, Z., Xuan, Z., Zhou, M., Liu, Q., & Qi, G.-J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00793 · Cited by 3
RAP: Retrieval-Augmented Planner for Adaptive Procedure Planning in Instructional Videos
Zare, A., Niu, Y., Ayyubi, H., & Chang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72980-5_24 · Cited by 3
S2CA: Shared Concept Prototypes and Concept-level Alignment for text–video retrieval
Li, Y., Yu, X., Qian, J., & Dong, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128851 · Cited by 3
Hierarchical Banzhaf Interaction for General Video-Language Representation Learning
Jin, P., Li, H., Yuan, L., Yan, S., & Chen, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3522124 · Cited by 3
Semantic structure preservation for accurate multi-modal glioma diagnosis
Shi, C., Zhang, X., Zhao, R., Zhang, W., & Chen, F. (2025) · Scientific Reports · doi:10.1038/s41598-025-88458-7 · Cited by 3
Learning a Grammar Inducer from Massive Uncurated Instructional Videos
Zhang, S., Song, L., Jin, L., Mi, H., Xu, K., Yu, D., & Luo, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.16 · Cited by 4
Active Learning for Video Description with Cluster-Regularized Ensemble Ranking
Chan, D. M., Vijayanarasimhan, S., Ross, D. A., & Canny, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69541-5_27 · Cited by 3
Enriching videos with automatic place recognition in google maps
Fallucchi, F., Stabile, R. D., Purificato, E., Giuliano, R., & Luca, E. W. D. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11253-9 · Cited by 3
Multi-modal Representation Learning for Video Advertisement Content Structuring
Guo, D., & Zeng, Z. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3479218 · Cited by 3
Dense Video Captioning for Incomplete Videos
Dang, X., Wang, G., Xiong, K., & Qin, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86383-8_53 · Cited by 3
A Multi -level Alignment Training Scheme for Video-and-Language Grounding
Zhang, Y., Niu, F., Ping, Q., & Thattai, G. (2022) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw58026.2022.00124 · Cited by 3
Nishimoto, S. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.08.22.457251 · Cited by 3
Shin, M., Mun, J., On, K.-W., Kang, W., Han, G., & Kim, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06476 · Cited by 4
Intuitive physics understanding emerges from self-supervised pretraining on natural videos
Garrido, Q., Ballas, N., Assran, M., Bardes, A., Najman, L., Rabbat, M., Dupoux, E., & LeCun, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.11831 · Cited by 3
Zero-shot Natural Language Video Localization
Nam, J., Ahn, D., Kang, D., Ha, S. J., & Choi, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00150 · Cited by 2
Mao, Y., Lin, X. A., Ni, Q., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i1.27814 · Cited by 2
He, T.-Y., Liu, H.-B., Ni, Z., Li, Y., Ma, X., Zhong, C., Zhang, Y., Wang, Y., & Lin, W.-Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02272-8 · Cited by 4
COM Kitchens: An Unedited Overhead-View Video Dataset as a Vision-Language Benchmark
Maeda, K., Hirasawa, T., Hashimoto, A., Harashima, J., Rybicki, L., Fukasawa, Y., & Ushiku, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73650-6_8 · Cited by 2
Video-Poetry Retrieval with Multimodal Knowledge Graph Guided Unsupervised Pre-training
Wei, X., Li, Y., & Wu, B. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890236 · Cited by 2
FIction: 4D Future Interaction Prediction from Video
Ashutosh, K., Pavlakos, G., & Grauman, K. L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01641 · Cited by 2
PRVR: Partially Relevant Video Retrieval
Chen, X., Liu, D., Yang, X., Li, X., Dong, J., Wang, M., & Wang, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3614169 · Cited by 2
Video is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generation
Waseem, F., & Shahzad, M. S. (2025) · ACM Computing Surveys · doi:10.1145/3771724 · Cited by 7
Metadata
- Authors
- Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2630-2640
- OpenAlex
- W2948859046
- Cited by
- 986
- References
- 69
- Citations by year
- 2026: 77 · 2025: 138 · 2024: 170 · 2023: 213 · 2022: 161 · 2021: 155 · 2020: 65 · 2019: 7
- FWCI
- 31.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- European Regional Development Fund (02.1.01/0.0/0.0, CZ.02, CZ.02.1.01, CZ.02.1.01/0.0, CZ.02.1.01/0.0/0.0)
- FP7 Ideas: European Research Council (336845)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video-text retrieval, action localization, domain transfer, large-scale video datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
