The DOI matches its Crossref record; 68 of the 69 references it cites were found in a registry or an index.
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Open accessAbstract
Learning text-video embeddings usually requires a dataset of video clips with manually provided captions. However, such datasets are expensive and time consuming to create and therefore difficult to obtain on a large scale. In this work, we propose instead to learn such embeddings from video data with readily available natural language annotations in the form of automatically transcribed narrations. The contributions of this work are three-fold. First, we introduce HowTo100M: a large-scale dataset of 136 million video clips sourced from 1.22M narrated instructional web videos depicting humans performing and describing over 23k different visual tasks. Our data collection procedure is fast, scalable and does not require any additional manual annotation. Second, we demonstrate that a text-video embedding trained on this data leads to state-of-the-art results for text-to-video retrieval and action localization on instructional video datasets such as YouCook2 or CrossTask. Finally, we show that this embedding transfers well to other domains: fine-tuning on generic Youtube videos (MSR-VTT dataset) and movies (LSMDC dataset) outperforms models trained on these datasets alone. Our dataset, code and models are publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The beginning and end of videos, especially tutorial videos (Miech et al., 2019), often consist of an intro, respectively an outro. These sections are not as related to the content of the videos and contain fewer movements and actions.”
From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Miech et al., 2019
Experimental — accuracy not yet measuredIn the citing paper
“More modalities.In addition to the general modalities of vision, text, and audio, further diverse modalities are emerging, e.g., Pano-AVQA [144] -the first large-scale spatial and audio-visual question answering dataset on 360 • videos, YouTube-360 (YT-360) [145] (360 • videos), AIST++ [146] (a new multimodal dataset of 3D dance motion and music), Artemis [147] (affective language for visual arts)”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [144]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?
Hara, K., Kataoka, H., & Satoh, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00685 · Cited by 2,346
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.571 · Cited by 1,888
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582
Unifying Visual-Semantic Embeddings with Multimodal Neural Language ModelsPreprint
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,496
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,279
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.494 · Cited by 1,199
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,125
Localizing Moments in Video with Natural Language
Hendricks, L. A., Wang, O., Shechtman, E., Šivic, J., Darrell, T., & Russell, B. C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.618 · Cited by 1,034
Sampling Matters in Deep Embedding Learning
Wu, C.-Y., Manmatha, R., Smola, A. J., & Krähenbühl, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.309 · Cited by 919
Scaling Egocentric Vision: The "Equation missing" Dataset
Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_44 · Cited by 911
Deep Fragment Embeddings for Bidirectional Image Sentence MappingPreprint
Karpathy, A., Joulin, A., & Fei-Fei, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.5679 · Cited by 825
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.541 · Cited by 795
Towards Automatic Learning of Procedures From Web Instructional Videos
Zhou, L., Xu, C., & Corso, J. J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12342 · Cited by 722
Sienkiewicz, H. (1895) · Cited by 669
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646
Self-citationAsk Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 632
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
Yu, H., Wang, J., Huang, Z., Yang, Y., & Xu, W. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.496 · Cited by 575
Jointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.497 · Cited by 572
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 549
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J. L., & Lazebnik, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 548
A dataset for Movie Description
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298940 · Cited by 457
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Pan, P., Xu, Z., Yang, Y., Wu, F. Q., & Zhuang, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.117 · Cited by 420
A Joint Sequence Fusion Model for Video Question Answering and Retrieval
Yu, Y., Kim, J.-S., & Kim, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_29 · Cited by 358
Rohrbach, A., Torabi, A., Rohrbach, M., Tandon, N., Pal, C., Larochelle, H., Courville, A. C., & Schiele, B. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0987-1 · Cited by 340
Learnable pooling with Context Gating for video classificationPreprint
Miech, A., Laptev, I., & Šivic, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.06905 · Cited by 320
Self-citationXu, R., Xiong, C., Chen, W. R., & Corso, J. J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9512 · Cited by 311
COIN: A Large-Scale Dataset for Comprehensive Instructional Video Analysis
Tang, Y., Ding, D., Rao, Y., Zheng, Y., Zhang, D., Zhao, L., Lu, J., & Zhou, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00130 · Cited by 300
Dual Encoding for Zero-Example Video Retrieval
Dong, J., Li, X., Xu, C., Ji, S., He, Y., Yang, G., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00957 · Cited by 300
Associating neural word embeddings with deep image representations using Fisher Vectors
Klein, B., Lev, G., Sadeh, G., & Wolf, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299073 · Cited by 292
Unsupervised Learning from Narrated Instruction Videos
Alayrac, J.-B., Bojanowski, P., Agrawal, N., Šivic, J., Ivan Laptev, A., & Lacoste-Julien, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.495 · Cited by 289
Self-citationImproving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections
Gong, Y., Wang, L., Hodosh, M., Hockenmaier, J., & Lazebnik, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_35 · Cited by 288
Learning Joint Embedding with Multimodal Cues for Cross-Modal Video-Text Retrieval
Mithun, N. C., Li, J., Metze, F., & Roy–Chowdhury, A. K. (2018) · International Conference on Multimedia Retrieval · doi:10.1145/3206025.3206064 · Cited by 250
Learning a Text-Video Embedding from Incomplete and Heterogeneous DataPreprint
Miech, A., Laptev, I., & Šivic, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02516 · Cited by 242
Self-citationTGIF: A New Dataset and Benchmark on Animated GIF Description
Li, Y., Song, Y., Cao, L., Tetreault, J., Goldberg, L., Jaimes, A., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.502 · Cited by 235
End-to-End Concept Word Detection for Video Captioning, Retrieval, and Question Answering
Yu, Y., Ko, H., Choi, J., & Kim, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.347 · Cited by 231
Cross-Task Weakly Supervised Learning From Instructional Videos
Zhukov, D., Alayrac, J.-B., Cinbiş, R. G., Fouhey, D. F., Laptev, I., & Šivic, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00365 · Cited by 229
Self-citationConnectionist Temporal Modeling for Weakly Supervised Action Labeling
Huang, D.-A., Fei-Fei, L., & Niebles, J. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_9 · Cited by 226
Towards Automatic Learning of Procedures from Web Instructional VideosPreprint
Zhou, L., Xu, C., & Corso, J. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09788 · Cited by 216
Weakly Supervised Action Learning with RNN Based Fine-to-Coarse Modeling
Richard, A., Kuehne, H., & Gall, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.140 · Cited by 195 · cited 1×
“Instructional videos are rising in popularity in the context of learning steps of complex tasks [2, 16, 41, 42, 46, 68], visual-linguistic reference resolution [17, 18], action segmentation in long untrimmed videos [66] and joint learning of object states and actions [3].”
BackgroundHow2: A Large-scale Dataset for Multimodal Language UnderstandingPreprint
Sanabria, R., Çağlayan, O., Palaskar, S., Elliott, D., Barrault, L., Specia, L., & Metze, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00347 · Cited by 179
Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input
Harwath, D. F., Recasens, A., Surís, D., Chuang, G., Torralba, A. B., & Glass, J. R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_40 · Cited by 177
What’s Cookin’? Interpreting Cooking Videos using Text, Speech and Vision
Malmaud, J., Huang, J., Rathod, V., Johnston, N., Rabinovich, A., & Murphy, K. P. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1015 · Cited by 159
Using Descriptive Video Services to Create a Large Data Source for Video Annotation ResearchPreprint
Torabi, A., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01070 · Cited by 149
Unsupervised Learning and Segmentation of Complex Activities from Video
Şener, F., & Yao, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00873 · Cited by 121
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?Preprint
Hara, K., Kataoka, H., & Satoh, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.09577 · Cited by 114
Enhancing Video Summarization via Vision-Language Embedding
Plummer, B. A., Brown, M., & Lazebnik, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.118 · Cited by 112
Unsupervised Semantic Parsing of Video Collections
Şener, O., Zamir, A., Savarese, S., & Saxena, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.509 · Cited by 89
Action Sets: Weakly Supervised Action Segmentation Without Ordering Constraints
Richard, A., Kuehne, H., & Gall, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00627 · Cited by 89
Finding "It": Weakly-Supervised Reference-Aware Visual Grounding in Instructional Videos
Huang, D.-A., Buch, S., Dery, L. M., Garg, A., Fei-Fei, L., & Niebles, J. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00623 · Cited by 89
Joint Discovery of Object States and Manipulation Actions
Alayrac, J.-B., Šivic, J., Laptev, I., & Lacoste-Julien, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.234 · Cited by 79
Self-citationWebly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval
Mithun, N. C., Panda, R., Papalexakis, E. E., & Roy–Chowdhury, A. K. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240712 · Cited by 79
Learning Language-Visual Embedding for Movie Understanding with Natural-LanguagePreprint
Torabi, A., Tandon, N., & Sigal, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08124 · Cited by 68
Instructional Videos for Unsupervised Harvesting and Learning of Action Examples
Yu, S.-I., Jiang, L., & Hauptmann, A. G. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654997 · Cited by 51
Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos
Huang, D.-A., Lim, J. J., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.116 · Cited by 49
Temporal Tessellation: A Unified Approach for Video Analysis
Kaufman, D., Levi, G., Hassner, T., & Wolf, L. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.20 · Cited by 37
Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning
Wang, X. E., Wu, J., Zhang, D., Su, Y., & Wang, W. Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018965 · Cited by 35
Video Captioning and Retrieval Models with Semantic Attention.Preprint
Yu, Y., Ko, H., Choi, J., & Kim, G. (2016) · arXiv (Cornell University) · Cited by 34
Discover and Learn New Objects from Documentaries
Chen, K., Song, H., Loy, C. C., & Lin, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.124 · Cited by 22
End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question AnsweringPreprint
Yu, Y., Ko, H., Choi, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02947 · Cited by 19
Research works printed but not linked: 4
- radford, 2018, Improving Language Understandingby Generative Pre-Training, Preprint
- 2019, Project webpage
- radford, 2019, Language models are unsupervised multitask learners, Preprint
- damen, 2018, Scaling egocentric vision: The epic-kitchens dataset, ECCV
Cited by
Sorted by citations
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Is Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966
Self-citation
Metadata
- Authors
- Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Institut national de recherche en sciences et technologies du numérique (FR)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 2630-2640
- OpenAlex
- W2948859046
- Cited by
- 986
- References
- 69
- Citations by year
- 2026: 77 · 2025: 138 · 2024: 170 · 2023: 213 · 2022: 161 · 2021: 155 · 2020: 65 · 2019: 7
- FWCI
- 31.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- European Regional Development Fund (02.1.01/0.0/0.0, CZ.02, CZ.02.1.01, CZ.02.1.01/0.0, CZ.02.1.01/0.0/0.0)
- FP7 Ideas: European Research Council (336845)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video-text retrieval, action localization, domain transfer, large-scale video datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
