CoChat
Sign Up
PreprintCited by 18Domain Adaptation and Few-Shot LearningHuman Pose and Action RecognitionMultimodal Machine Learning Applications

Revisiting Feature Prediction for Learning Visual Representations from Video

Open access
Bardes, Adrien, Garrido, Quentin, Ponce, Jean, Chen, Xinlei, Rabbat, Michael, LeCun, Yann, Assran, Mahmoud, Ballas, Nicolas·2024·arXiv
DOI 10.48550/arxiv.2404.08471arXiv 2404.08471
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper explores feature prediction as a stand-alone objective for unsupervised learning from video and introduces V-JEPA, a collection of vision models trained solely using a feature prediction objective, without the use of pretrained image encoders, text, negative examples, reconstruction, or other sources of supervision. The models are trained on 2 million videos collected from public datasets and are evaluated on downstream image and video tasks. Our results show that learning by predicting video features leads to versatile visual representations that perform well on both motion and appearance-based tasks, without adaption of the model's parameters; e.g., using a frozen backbone. Our largest model, a ViT-H/16 trained only on videos, obtains 81.9% on Kinetics-400, 72.2% on Something-Something-v2, and 77.9% on ImageNet1K.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2404.08471

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Maintaining fixed teacher EMA and weight decay coefficients instead of using a ramp-up schedule showed minimal impact on downstream understanding tasks.

    “We simplified the recipe from Bardes et al. (2024) by maintaining fixed teacher EMA and weight decay coefficients instead of using ramp-up schedule, as these variations showed minimal impact on downstream understanding tasks.”

    From V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025) · cites this paper as Bardes et al., 2024

    The source’s Pretraining details (Appendix C; page not provided) confirm that Bardes et al. increased both teacher EMA momentum and weight decay during training. They report that truncating those schedules improved performance, but do not compare fixed coefficients with scheduled ones on downstream tasks. The claimed minimal impact of that change is therefore not supported by this source. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Optimizing the context size per property showcases the maximal attainable performance and evaluates whether the model has the necessary information or capabilities to solve the task at hand.

    “We thus opted to showcase the maximal attainable performance by optimizing the context size per property, as is commonly done in the self-supervised learning literature (Bardes et al., 2024). This means that we are evaluating whether the model has the necessary information or capabilities to solve the task at hand.”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Bardes et al., 2024

    Section D.1 (page unavailable) uses different numbers of video clips for two downstream tasks, and Appendix E.1 reports that greater temporal coverage improves K400 performance. This supports varying evaluation context by task, but the paper does not establish that context size was optimized for each property to show maximal attainable performance. The cited paper also does not make the citing authors’ subsequent claim about testing whether a model has the necessary information or capabilities. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • The prediction task is not as important for intuitive physics understanding.

    “Contrary to classical video tasks (Bardes et al., 2024), we find that the prediction task is not as important for intuitive physics understanding (see figure 3.B).”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Bardes et al., 2024

  • The same evaluation protocol was used for every baseline and for V-JEPA 2, learning an attentive probe on top of the frozen encoder.

    “We use the same evaluation protocol for every baseline and for V-JEPA 2, learning an attentive probe on top of the frozen encoder, similar to Bardes et al. (2024).”

    From V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025) · cites this paper as Bardes et al. (2024)

  • JEPA-Anything predicts target representations from observed context, building on joint-embedding predictive architectures.

    “Building on joint-embedding predictive architectures [10-12], it predicts target representations from observed context.”

    From RSI-Anything: Unlocking Recursive Self-Improvement with World Models (2026) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “ViT-L/16 56.7 73.7 50.1 66.2 has been critical for enabling the surge of advancements in other modalities, such as text and images (Kaplan et al., 2020; Cherti et al., 2023).”
    BackgroundInfluential

Flagged first, then by citations

Research works printed but not linked: 77

  1. Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, and Boqing Gong. Vatt: Trans- formers for multimodal self-supervised learning from raw video, audio and text. Advances in Neural Information Processing Systems, 34:24206-24221, 2021.
  2. Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lucic, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE international conference on computer vision, 2021.
  3. Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Michael Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. arXiv preprint arXiv:2204.07141, 2022. · arXiv
  4. Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bo- janowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619-15629, 2023.
  5. Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael Auli. Efficient self-supervised learning with contextualized target representations for vision, speech and language. arXiv preprint arXiv:2212.07525, 2022a. · arXiv
  6. Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. Data2vec: A general frame- work for self-supervised learning in speech, vision and language. arXiv preprint arXiv:2202.03555, 2022b. · arXiv
  7. Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. · arXiv
  8. Pietro Berkes and Laurenz Wiskott. Slow feature analysis yields a rich repertoire of complex cell properties. Journal of vision, 5(6):9-9, 2005.
  9. Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learn- ing of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.09882, 2020. · arXiv
  10. Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jé- gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. arXiv preprint arXiv:2104.14294, 2021. · arXiv
  11. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. preprint arXiv:2002.05709, 2020. · arXiv
  12. Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin, Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo, Gang Zeng, and Jingdong Wang. Context autoencoder for self-supervised representation learning. arXiv preprint arXiv:2202.03026, 2022. · arXiv
  13. Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. arXiv preprint arXiv:2104.02057, 2021. · arXiv
  14. Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Repro- ducible scaling laws for contrastive language-image learn- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 2818-2829, 2023.
  15. Ekin Dogus Cubuk, Barret Zoph, Vijay Mane, Dandelion and- Vasudevan, and Quoc V. Le. Autoaugment: Learning aug- mentation policies from data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  16. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. · arXiv
  17. Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Girshick, and Kaiming He. A large-scale study on unsupervised spa- tiotemporal representation learning. Proceedings of the IEEE conference on computer vision and pattern recogni- tion, 2021.
  18. Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. Masked autoencoders as spatiotemporal learners. Advances in neural information processing systems, 35:35946-35958, 2022.
  19. David J Field. What is the goal of sensory coding? Neural computation, 6(4):559-601, 1994.
  20. Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, and Matthieu Cord. Learning representations by predicting bags of visual words. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6928-6938, 2020.
  21. Rohit Girdhar and Kristen Grauman. Anticipative video transformer. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pages 13505-13515, 2021.
  22. Rohit Girdhar, Alaaeldin El-Nouby, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Omnimae: Single model masked pretraining on images and videos. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 10406-10417, 2023.
  23. Ross Goroshin, Joan Bruna, Jonathan Tompson, David Eigen, and Yann LeCun. Unsupervised learning of spatiotempo- rally coherent metrics. In Proceedings of the IEEE inter- national conference on computer vision, pages 4086-4093, 2015.
  24. Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michal- ski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pages 5842-5850, 2017.
  25. Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Do- ersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Moham- mad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733, 2020. · arXiv
  26. Chunhui Gu, Chen Sun, David A Ross, Carl Vondrick, Caro- line Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, et al. Ava: A video dataset of spatio-temporally localized atomic visual actions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6047-6056, 2018.
  27. Agrim Gupta, Jiajun Wu, Jia Deng, and Li Fei-Fei. Siamese masked autoencoders. arXiv preprint arXiv:2305.14344, 2023. · arXiv
  28. Michael U Gutmann and Aapo Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with appli- cations to natural image statistics. Journal of machine learning research, 13(2), 2012.
  29. Tengda Han, Weidi Xie, and Andrew Zisserman. Video representation learning by dense predictive coding. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0-0, 2019.
  30. Tengda Han, Weidi Xie, and Andrew Zisserman. Memory- augmented dense predictive coding for video representation learning. In European conference on computer vision, pages 312-329. Springer, 2020.
  31. Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dol- lár, and Ross Girshick. Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377, 2021. · arXiv
  32. Geoffrey E Hinton. Connectionist learning procedures. In Machine learning, pages 555-610. Elsevier, 1989.
  33. Tarun Kalluri, Deepak Pathak, Manmohan Chandraker, and Du Tran. Flavr: Flow-agnostic video representations for fast frame interpolation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2071-2082, 2023.
  34. Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Vi- ola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017. · arXiv
  35. Christoph Kayser, Wolfgang Einhäuser, Olaf Dümmer, Peter König, and Konrad Körding. Extracting slow subspaces from natural videos leads to complex cells. In Artificial Neural Networks-ICANN 2001: International Conference Vienna, Austria, August 21-25, 2001 Proceedings 11, pages 1075-1080. Springer, 2001.
  36. Hsin-Ying Lee, Jia-Bin Huang, Maneesh Singh, and Ming- Hsuan Yang. Unsupervised representation learning by sorting sequences. In Proceedings of the IEEE international conference on computer vision, pages 667-676, 2017.
  37. Kunchang Li, Yali Wang, Peng Gao, Guanglu Song, Yu Liu, Hongsheng Li, and Yu Qiao. Uniformer: Unified trans- former for efficient spatiotemporal representation learning. arXiv preprint arXiv:2201.04676, 2022. · arXiv
  38. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. · arXiv
  39. Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. Howto100m: Learning a text-video embedding by watch- ing hundred million narrated video clips. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2630-2640, 2019.
  40. Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Euro- pean conference on computer vision, pages 69-84. Springer, 2016.
  41. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Represen- tation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. · arXiv
  42. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervi- sion. arXiv preprint arXiv:2304.07193, 2023. · arXiv
  43. Nikhil Parthasarathy, SM Eslami, João Carreira, and Olivier J Hénaff. Self-supervised video pretraining yields strong image representations. arXiv preprint arXiv:2210.06433, 2022. · arXiv
  44. Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 2536-2544, 2016.
  45. Silvia L Pintea, Jan C van Gemert, and Arnold WM Smeul- ders. Déja vu: Motion prediction in static images. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part III 13, pages 172-187. Springer, 2014.
  46. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language su- pervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
  47. Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999.
  48. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition chal- lenge. International Journal of Computer Vision, 115(3): 211-252, 2015.
  49. Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, Chen Wei, Haoqi Fan, Po-Yao Huang, Vaibhav Aggarwal, Arka- bandhu Chowdhury, Omid Poursaeed, Judy Hoffman, et al. Hiera: A hierarchical vision transformer without the bells- and-whistles. arXiv preprint arXiv:2306.00989, 2023. · arXiv
  50. Laura Sevilla-Lara, Shengxin Zha, Zhicheng Yan, Vedanuj Goswami, Matt Feiszli, and Lorenzo Torresani. Only time can tell: Discovering temporal data for temporal modeling. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 535-544, 2021.
  51. Nitish Srivastava, Elman Mansimov, and Ruslan Salakhudi- nov. Unsupervised learning of video representations using lstms. In International conference on machine learning, pages 843-852. PMLR, 2015.
  52. Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7464-7473, 2019.
  53. Dídac Surís, Ruoshi Liu, and Carl Vondrick. Learning the pre- dictability of the future. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12607-12617, 2021.
  54. Reuben Tan, Matthias De Lange, Michael Iuzzolino, Bryan A Plummer, Kate Saenko, Karl Ridgeway, and Lorenzo Tor- resani. Multiscale video pretraining for long-term activity forecasting. arXiv preprint arXiv:2307.12854, 2023. · arXiv
  55. Antti Tarvainen and Harri Valpola. Mean teachers are bet- ter role models: Weight-averaged consistency targets im- prove semi-supervised deep learning results. arXiv preprint arXiv:1703.01780, 2017. · arXiv
  56. Yuandong Tian, Xinlei Chen, and Surya Ganguli. Under- standing self-supervised learning dynamics without con- trastive pairs. In International Conference on Machine Learning, pages 10268-10278. PMLR, 2021.
  57. Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Video- mae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078-10093, 2022.
  58. Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 8769-8778, 2018.
  59. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre- Antoine Manzagol. Extracting and composing robust fea- tures with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, page 1096-1103, 2008.
  60. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Ben- gio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010.
  61. Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. Anticipating visual representations from unlabeled video. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 98-106, 2016.
  62. Fei Wang, Ping Li, and Arnd Christian Konig. Learning a bi-stochastic data similarity matrix. In 2010 IEEE International Conference on Data Mining, pages 551-560. IEEE, 2010.
  63. Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14549-14560, 2023a.
  64. Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Lu Yuan, and Yu-Gang Jiang. Masked video distillation: Rethinking masked feature mod- eling for self-supervised video representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6312-6322, 2023b.
  65. Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint arXiv:2212.03191, 2022. · arXiv
  66. Laurenz Wiskott and Terrence J Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural computation, 14(4):715-770, 2002.
  67. Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3733-3742, 2018.
  68. Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A sim- ple framework for masked image modeling. arXiv preprint arXiv:2111.09886, 2021. · arXiv
  69. Dejing Xu, Jun Xiao, Zhou Zhao, Jian Shao, Di Xie, and Yueting Zhuang. Self-supervised spatiotemporal learn- ing via video clip order prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10334-10343, 2019.
  70. Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. Videoclip: Contrastive pre- training for zero-shot video-text understanding. arXiv preprint arXiv:2109.14084, 2021. · arXiv
  71. Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022. · arXiv
  72. Liangzhe Yuan, Nitesh Bharadwaj Gundavarapu, Long Zhao, Hao Zhou, Yin Cui, Lu Jiang, Xuan Yang, Menglin Jia, Tobias Weyand, Luke Friedman, et al. Videoglue: Video general understanding evaluation of foundation models. arXiv preprint arXiv:2307.03166, 2023. · arXiv
  73. Rowan Zellers, Jiasen Lu, Ximing Lu, Youngjae Yu, Yanpeng Zhao, Mohammadreza Salehi, Aditya Kusupati, Jack Hes- sel, Ali Farhadi, and Yejin Choi. Merlot reserve: Neural script knowledge through vision and language and sound. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16375-16387, 2022.
  74. Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Tor- ralba, and Aude Oliva. Learning deep features for scene recognition using places database. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Wein- berger, editors, Advances in Neural Information Pro- cessing Systems, volume 27.
  75. Curran Associates, Inc., 2014. https://proceedings.neurips.cc/paper/2014/file/ 3fe94a002317b5f9259f82690aeea4cd-Paper.pdf.
  76. Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. Ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832, 2021. · arXiv
  77. Will Zou, Shenghuo Zhu, Kai Yu, and Andrew Ng. Deep learning of invariant features via simulated fixations in video. Advances in neural information processing systems, 25, 2012.

Cited by

Sorted by citations

Authors
Bardes, A., Garrido, Q., Ponce, J., Chen, X., Rabbat, M., LeCun, Y., Assran, M., & Ballas, N.
Affiliations
  • Institut national de recherche en sciences et technologies du numérique (FR)
  • Université Gustave Eiffel (FR)
  • New York University (US)
  • Centre National de la Recherche Scientifique (FR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2404.08471
arXiv
2404.08471
OpenAlex
W4394861491
Cited by
18
References
1
Citations by year
2026: 3 · 2025: 6 · 2024: 9
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
feature prediction, visual representation learning, vision transformer, frozen backbone, Kinetics-400, Something-Something V2, ImageNet-1K

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review