A large-scale study on unsupervised spatiotemporal representation learning
Open accessAbstract
We present a large-scale study on unsupervised spatiotemporal representation learning from videos. With a unified perspective on four recent image-based frameworks, we study a simple objective that can easily generalize all these methods to space-time. Our objective encourages temporally-persistent features in the same video, and in spite of its simplicity, it works surprisingly well across: (i) different unsupervised frameworks, (ii) pre-training datasets, (iii) downstream datasets, and (iv) backbone architectures. We draw a series of intriguing observations from this study, e.g., we discover that encouraging long-spanned persistency can be effective even if the timespan is 60 seconds. In addition to state-of-the-art results in multiple benchmarks, we report a few promising cases in which unsupervised pre-training can outperform its supervised counterpart. Code will be made available at https://github.com/facebookresearch/SlowFast.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Self-citationVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Self-citationHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Self-citationGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Self-citationBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Self-citationA simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
(2011) · Cited by 6,993 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575 · see citing works
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · see citing works
Self-citationHmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965 · see citing works
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,803 · see citing works
The Kinetics Human Action Video DatasetPreprint
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06950 · Cited by 3,711 · see citing works
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672 · see citing works
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664 · see citing works
Audio Set: An ontology and human-labeled dataset for audio events
Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952261 · Cited by 3,134 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Improved Baselines with Momentum Contrastive LearningPreprint
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 2,579 · see citing works
Self-citationUnsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499 · see citing works
Sinkhorn Distances: Lightspeed Computation of Optimal TransportPreprint
Cuturi, M. (2013) · Cited by 2,385 · see citing works
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · see citing works
Self-citationTSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,152 · see citing works
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997 · see citing works
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., & Bengio, Y. (2018) · arXiv · doi:10.48550/arxiv.1808.06670 · Cited by 1,991 · see citing works
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861 · see citing works
The kinetics human action video dataset
Kay, W. (2024) · TIB Data Manager · doi:10.57702/jtuxnt5m · Cited by 1,646 · see citing works
Big Self-Supervised Models are Strong Semi-Supervised LearnersPreprint
Chen, T., Kornblith, S., Swersky, K., Norouzi, M., & Hinton, G. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10029 · Cited by 1,480 · see citing works
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471 · see citing works
Slow Feature Analysis: Unsupervised Learning of Invariances
Wiskott, L., & B, T. J. S. A. (2002) · Neural Computation · doi:10.1162/089976602317318938 · Cited by 1,351 · see citing works
Deep multi-scale video prediction beyond mean square errorPreprint
Mathieu, M., Couprie, C., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05440 · Cited by 1,182 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,098 · see citing works
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Gu, C., Sun, C., Ross, D. A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan, S., Toderici, G. D., Ricco, S., Sukthankar, R., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00633 · Cited by 1,029 · see citing works
Data-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014 · see citing works
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005 · see citing works
Large Batch Training of Convolutional NetworksPreprint
You, Y., Gitman, I., & Ginsburg, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03888 · Cited by 992 · see citing works
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986 · see citing works
Arandjelović, R., & Zisserman, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 836 · see citing works
Deep Predictive Coding Networks for Video Prediction and Unsupervised LearningPreprint
Lotter, W. E., Kreiman, G., & Cox, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08104 · Cited by 805 · see citing works
Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification
Misra, I., Zitnick, C. L., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_32 · Cited by 779 · see citing works
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks
Dosovitskiy, A., Fischer, P., Springenberg, J. T., Riedmiller, M., & Brox, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2496141 · Cited by 762 · see citing works
Agrawal, P., Carreira, J., & Malik, J. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.13 · Cited by 639 · see citing works
Time-Contrastive Networks: Self-Supervised Learning from Video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Stefan Schaal, A., Levine, S., & Brain, G. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462891 · Cited by 579 · see citing works
Learning Features by Watching Objects Move
Pathak, D., Girshick, R. B., Dollár, P., Darrell, T., & Hariharan, B. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.638 · Cited by 549 · see citing works
Self-citationAnticipating Visual Representations from Unlabeled Video
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.18 · Cited by 538 · see citing works
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · arXiv · doi:10.48550/arxiv.1807.05520 · Cited by 533 · see citing works
Unsupervised Representation Learning by Sorting Sequences
Lee, H.-Y., Huang, J., Singh, M., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.79 · Cited by 529 · see citing works
Spatiotemporal Contrastive Video Representation Learning
Qian, R., Meng, T., Gong, B., Yang, M., Wang, H., Belongie, S., & Cui, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00689 · Cited by 462 · see citing works
Learning Correspondence From the Cycle-Consistency of Time
Wang, X., Jabri, A., & Efros, A. A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00267 · Cited by 452 · see citing works
Arandjelović, R., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_27 · Cited by 448 · see citing works
Local Aggregation for Unsupervised Learning of Visual Embeddings
Zhuang, C., Zhai, A., & Yamins, D. L. K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00610 · Cited by 422 · see citing works
Self-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421 · see citing works
Self-Supervised Spatiotemporal Learning via Video Clip Order Prediction
Xu, D., Xiao, J., Zhao, Z., Shao, J., Xie, D., & Zhuang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01058 · Cited by 417 · see citing works
Video Representation Learning by Dense Predictive Coding
Han, T., Xie, W., & Zisserman, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00186 · Cited by 383 · see citing works
Owens, A., Isola, P., McDermott, J. H., Torralba, A. B., Adelson, E. H., & Freeman, W. T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.264 · Cited by 352 · see citing works
A Short Note on the Kinetics-700-2020 Human Action Dataset
Smaira, L., Carreira, J., Noland, E., Clancy, E., Wu, A., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.10864 · Cited by 334 · see citing works
Cooperative Learning of Audio and Video Models from Self-Supervised SynchronizationPreprint
Korbar, B., Tran, D., & Torresani, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00230 · Cited by 331 · see citing works
Deep learning from temporal coherence in video
Mobahi, H., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553469 · Cited by 328 · see citing works
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Kim, D., Cho, D., & Kweon, I. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018545 · Cited by 314 · see citing works
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298 · see citing works
Self-supervised Co-training for Video Representation Learning
Han, T., Xie, W., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.09709 · Cited by 292 · see citing works
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition
Ghadiyaram, D., Tran, D., & Mahajan, D. K. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01232 · Cited by 286 · see citing works
A Short Note about Kinetics-600
Carreira, J., Noland, E., Banki-Horvath, A., Hillier, C., & Zisserman, A. (2018) · arXiv · doi:10.48550/arxiv.1808.01340 · Cited by 270 · see citing works
Self-Supervised MultiModal Versatile NetworksPreprint
Alayrac, J.-B., Recasens, A., Schneider, R. G., Arandjelović, R., Ramapuram, J., Fauw, J. D., Smaira, L., Dieleman, S., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16228 · Cited by 247 · see citing works
SpeedNet: Learning the Speediness in Videos
Benaim, S., Ephrat, A., Lang, O., Mosseri, I., Freeman, W. T., Rubinstein, M., Irani, M., & Dekel, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00994 · Cited by 234 · see citing works
Learning Image Representations Tied to Ego-Motion
Jayaraman, D., & Grauman, K. L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.166 · Cited by 229 · see citing works
Hollywood in Homes: Crowdsourcing Data Collection for Activity UnderstandingPreprint
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01753 · Cited by 221 · see citing works
Audio-Visual Instance Discrimination with Cross-Modal Agreement
Morgado, P. M., Vasconcelos, N. M., & Misra, I. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01229 · Cited by 218 · see citing works
Rethinking Spatiotemporal Feature Learning For Video Understanding.
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. P. (2017) · Cited by 205 · see citing works
Unsupervised Feature Learning via Non-Parametric Instance-level DiscriminationPreprint
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01978 · Cited by 189 · see citing works
Transitive Invariance for Self-Supervised Visual Representation Learning
Wang, X., He, K., & Gupta, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.149 · Cited by 173 · see citing works
Self-citationUnsupervised Learning of Spatiotemporally Coherent Metrics
Goroshin, R., Bruna, J., Tompson, J., Eigen, D., & LeCun, Y. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.465 · Cited by 156 · see citing works
Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset BiasesPreprint
Purushwalkam, S., & Gupta, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13916 · Cited by 137 · see citing works
Evolving Losses for Unsupervised Video Representation Learning
Piergiovanni, A., Angelova, A., & Ryoo, M. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00021 · Cited by 134 · see citing works
Video Representation Learning by Recognizing Temporal Transformations
Jenni, S., Meishvili, G., & Favaro, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58604-1_26 · Cited by 126 · see citing works
Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning
Gan, C., Gong, B., Liu, K., Su, H., & Guibas, L. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00586 · Cited by 124 · see citing works
Multi-modal Self-Supervision from Generalized Data TransformationsPreprint
Patrick, M., Asano, Y. M., Кузнецова, П., Fong, R., Henriques, J. F., Zweig, G., & Vedaldi, A. (2021) · arXiv (Cornell University) · Cited by 117 · see citing works
DynamoNet: Dynamic Action and Motion Network
Diba, A., Sharma, V., Gool, L. V., & Stiefelhagen, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00629 · Cited by 111 · see citing works
Contrastive Bidirectional Transformer for Temporal Representation Learning
Sun, C., Baradel, F., Murphy, K. P., & Schmid, C. (2019) · Cited by 109 · see citing works
Learning visual groups from co-occurrences in space and time
Isola, P., Zoran, D., Krishnan, D., & Adelson, E. H. (2015) · arXiv · doi:10.48550/arxiv.1511.06811 · Cited by 90 · see citing works
The AVA-Kinetics Localized Human Actions Video DatasetPreprint
Li, A., Thotakuri, M., Ross, D. A., Carreira, J., Vostrikov, A., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00214 · Cited by 83 · see citing works
Video Representation Learning with Visual Tempo ConsistencyPreprint
Yang, C., Xu, Y., Dai, B., & Zhou, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15489 · Cited by 64 · see citing works
Watching the World Go By: Representation Learning from Unlabeled VideosPreprint
Gordon, D., Ehsani, K., Fox, D., & Farhadi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07990 · Cited by 63 · see citing works
BYOL works even without batch statistics
Richemond, P. H., Grill, J.-B., Altché, F., Tallec, C., Strub, F., Brock, A., Smith, S., De, S., Pascanu, R., Piot, B., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.10241 · Cited by 51 · see citing works
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D. K., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2020) · King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) · Cited by 42 · see citing works
Learning Spatiotemporal Features via Video and Text Pair DiscriminationPreprint
Li, T., & Wang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05691 · Cited by 35 · see citing works
Learning Temporally Persistent Hierarchical Representations
Becker, S. (1996) · neural information processing systems · Cited by 11 · see citing works
Deep neural networks in computer vision and biomedical image analysis
Xie, W. (2017) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-gamgyb4j9 · Cited by 1 · see citing works
Flagged first, then by citations
6 of the 104 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- R. B. Girshick. Fast R-CNN. In Proc. ICCV, 2015. 12
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask R-CNN. In Proc. ICCV, 2017. 13
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level perfor- mance on imagenet classification. In Proc. ICCV, 2015. 11
- Xu Ji, João F. Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In Proc. ICCV, pages 9865-9874, 2019. 2
- Andrew Owens, Phillip Isola, Josh H. McDermott, Antonio Torralba, Edward H. Adelson, and William T. Freeman. Vi- sually indicated sounds. In Proc. CVPR, pages 2405-2413, 2016. 2
- Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. An- ticipating visual representations from unlabelled video. In Proc. CVPR, 2016. 2
Cited by
Sorted by citations
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 622 · see citing works
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587 · see citing works
Self-citationVideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909 · see citing works
BEVT: BERT Pretraining of Video Transformers
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Jiang, Y.-G., Zhou, L., & Yuan, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01432 · Cited by 234 · see citing works
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
Chen, Y., Mancini, M., Zhu, X., & Akata, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3201576 · Cited by 235 · see citing works
Frozen CLIP Models are Efficient Video Learners
Lin, Z., Geng, S., Zhang, R., Gao, P., de Melo, G., Wang, X., Dai, J., Qiao, Y., & Li, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19833-5_23 · Cited by 213 · see citing works
A Cookbook of Self-Supervised Learning
Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bordes, F., Bardes, A., Mialon, G., Tian, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12210 · Cited by 182 · see citing works
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Haliassos, A., Mira, R., Petridis, S., & Pantic, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01453 · Cited by 180 · see citing works
Selva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 177 · see citing works
TCLR: Temporal contrastive learning for video representation
Dave, I. R., Gupta, R., Rizve, M. N., & Shah, M. A. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103406 · Cited by 165 · see citing works
Shu, X., Xu, B., Zhang, L., & Tang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3222871 · Cited by 159 · see citing works
Unsupervised Point Cloud Representation Learning With Deep Neural Networks: A Survey
Xiao, A., Huang, J., Guan, D., Zhang, X., Lu, S., & Shao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262786 · Cited by 162 · see citing works
Self-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155 · see citing works
SignBERT+: Hand-Model-Aware Self-Supervised Pre-Training for Sign Language Understanding
Hu, H., Zhao, W., Zhou, W., & Li, H. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3269220 · Cited by 154 · see citing works
SVFormer: Semi-supervised Video Transformer for Action Recognition
Xing, Z., Dai, Q., Han, H., Chen, J., Wu, Z., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01804 · Cited by 126 · see citing works
Selective Structured State-Spaces for Long-Form Video Understanding
Wang, J., Zhu, W., Wang, P., Yu, X., Liu, L. L., Omar, M., & Hamid, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00618 · Cited by 121 · see citing works
Self-supervised Video Transformer
Ranasinghe, K., Naseer, M., Khan, S. H., Khan, F. S., & Ryoo, M. S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00289 · Cited by 123 · see citing works
A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions
Zhou, S., Xu, H., Zheng, Z., Chen, J., Li, Z., Bu, J., Wu, J. X., Wang, X. Y., Zhu, W., & Ester, M. (2024) · ACM Computing Surveys · doi:10.1145/3689036 · Cited by 209 · see citing works
Broaden Your Views for Self-Supervised Video Learning
Recasens, A., Luc, P., Alayrac, J.-B., Wang, L., Strub, F., Tallec, C., Malinowski, M., Pătrăucean, V., Altché, F., Valko, M., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00129 · Cited by 90 · see citing works
Wang, R., Chen, D., Wu, Z., Chen, Y., Dai, X., Liu, M., Yuan, L., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00611 · Cited by 84 · see citing works
Learning from Temporal Gradient for Semi-supervised Action Recognition
Xiao, J., Jing, L., Zhang, L., He, J., She, Q., Zhou, Z., Yuille, A., & Li, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00325 · Cited by 84 · see citing works
SPAct: Self-supervised Privacy Preservation for Action Recognition
Dave, I. R., Chen, C., & Shah, M. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01953 · Cited by 101 · see citing works
Deep learning serves traffic safety analysis: A forward‐looking review
Razi, A., Chen, X., Li, H., Wang, H., Russo, B. J., Chen, Y., & Yu, H. (2022) · IET Intelligent Transport Systems · doi:10.1049/itr2.12257 · Cited by 77 · see citing works
Dissecting self-supervised learning methods for surgical computer vision
Ramesh, S., Srivastav, V., Alapatt, D., Yu, T., Murali, A., Sestini, L., Nwoye, C. I., Hamoud, I., Sharma, S., Fleurentin, A., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102844 · Cited by 70 · see citing works
Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition
Xu, Y., Wei, F., Sun, X. N., Yang, C., Shen, Y., Dai, B., Zhou, B., & Lin, S. C.-F. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00297 · Cited by 64 · see citing works
Learning To Recognize Procedural Activities with Distant Supervision
Lin, X., Petroni, F., Bertasius, G., Rohrbach, M., Chang, S., & Torresani, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01348 · Cited by 62 · see citing works
Motion-aware Contrastive Video Representation Learning via Foreground-background Merging
Ding, S., Li, M., Yang, T., Qian, R., Xu, H., Chen, Q., Wang, J., & Xiong, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00949 · Cited by 63 · see citing works
C2F-TCN: A Framework for Semi- and Fully-Supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284080 · Cited by 60 · see citing works
Bakhtiari, S., Mineault, P., Lillicrap, T., Pack, C. C., & Richards, B. A. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.18.448989 · Cited by 60 · see citing works
Probabilistic Representations for Video Contrastive Learning
Park, J., Lee, J., Kim, I.-J., & Sohn, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01430 · Cited by 59 · see citing works
Learning Gait Representation From Massive Unlabelled Walking Videos: A Benchmark
Fan, C., Hou, S., Wang, J., Huang, Y., & Yu, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3312419 · Cited by 58 · see citing works
MAR: M asked Autoencoders for Efficient A ction R ecognition
Qing, Z., Zhang, S., Huang, Z., Wang, X., Wang, Y., Lv, Y., Gao, C., & Sang, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3263288 · Cited by 72 · see citing works
Contrast and Order Representations for Video Self-supervised Learning
Hu, K., Shao, J., Liu, Y., Raj, B., Savvides, M., & Shen, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00784 · Cited by 55 · see citing works
Diffusion Models as Masked Autoencoders
Wei, C., Mangalam, K., Huang, P.-Y., Li, Y., Fan, H., Hu, X., Wang, H., Xie, C., Yuille, A., & Feichtenhofer, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01492 · Cited by 58 · see citing works
Self-citationFan, H., Murrell, T., Wang, H., Alwala, K. V., Li, Y., Li, Y., Xiong, B., Ravi, N., Li, M., Yang, H., et al. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3478329 · Cited by 50 · see citing works
Self-citationLearnable Locality-Sensitive Hashing for Video Anomaly Detection
Lu, Y., Cao, C., Zhang, Y., & Zhang, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3205348 · Cited by 50 · see citing works
An Improved Inter-Intra Contrastive Learning Framework on Self-Supervised Video Representation
Tao, L., Wang, X., & Yamasaki, T. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3141051 · Cited by 50 · see citing works
Fan, L., Sowmya, A., Meijering, E., & Song, Y. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3228275 · Cited by 54 · see citing works
Temporal augmented contrastive learning for micro-expression recognition
Wang, T., & Shang, L. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.02.003 · Cited by 46 · see citing works
Masked Motion Encoding for Self-Supervised Video Representation Learning
Sun, X., Chen, P., Chen, L., Li, C., Li, T. H., Tan, M., & Gan, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00222 · Cited by 41 · see citing works
Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning
Chen, M., Wei, F., Li, C., & Cai, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01343 · Cited by 41 · see citing works
Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives
Hoffmann, D. T., Behrmann, N., Gall, J., Brox, T., & Noroozi, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19972 · Cited by 40 · see citing works
CONVIQT: Contrastive Video Quality Estimator
Madhusudana, P. C., Birkbeck, N., Wang, Y.-L., Adsumilli, B., & Bovik, A. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3310344 · Cited by 41 · see citing works
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning
Qing, Z., Zhang, S., Huang, Z., Zhang, Y., Gao, C., Zhao, D., & Sang, N. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01281 · Cited by 38 · see citing works
Neighbor-Guided Consistent and Contrastive Learning for Semi-Supervised Action Recognition
Wu, J., Sun, W., Gan, T., Ding, N., Jiang, F., Shen, J., & Nie, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3265261 · Cited by 34 · see citing works
SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition
Chappa, N. V. S. R., Nguyen, P., Nelson, A., Seo, H., Li, X., Dobbs, P. D., & Luu, K. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00544 · Cited by 35 · see citing works
Reformulating Graph Kernels for Self-Supervised Space-Time Correspondence Learning
Qin, Z., Lu, X., Liu, D., Nie, X., Yin, Y., Shen, J., & Loui, A. C. P. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3328485 · Cited by 33 · see citing works
SVFAP: Self-Supervised Video Facial Affect Perceiver
Sun, L., Lian, Z., Wang, K., He, Y., Xu, M., Sun, H., Liu, B., & Tao, J. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3436913 · Cited by 31 · see citing works
Improving self-supervised action recognition from extremely augmented skeleton sequences
Guo, T.-Y., Liu, M.-Y., Liu, H., Wang, G.-Q., & Li, W.-H. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110333 · Cited by 31 · see citing works
Iterative Contrast-Classify for Semi-supervised Temporal Action Segmentation
Singhania, D., Rahaman, R., & Yao, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20124 · Cited by 27 · see citing works
Contextualized Spatio-Temporal Contrastive Learning with Self-Supervision
Yuan, L., Qian, R., Cui, Y., Gong, B., Schroff, F., Yang, M., Adam, H., & Liu, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01359 · Cited by 27 · see citing works
VicTR: Video-conditioned Text Representations for Activity Recognition
Kahatapitiya, K., Arnab, A., Nagran, A., & Ryoo, M. S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01755 · Cited by 26 · see citing works
Static and Dynamic Concepts for Self-supervised Video Representation Learning
Qian, R., Ding, S., Liu, X., & Lin, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_9 · Cited by 26 · see citing works
ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints
Das, S., & Ryoo, M. S. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00553 · Cited by 26 · see citing works
XKD: Cross-Modal Knowledge Distillation with Domain Alignment for Video Representation Learning
Sarkar, P., & Etemad, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29407 · Cited by 26 · see citing works
Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency
Wu, H., & Wang, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00999 · Cited by 25 · see citing works
A Semantic and Motion-Aware Spatiotemporal Transformer Network for Action Detection
Korban, M., Youngs, P. A., & Acton, S. T. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3377192 · Cited by 25 · see citing works
Self-Supervised Contrastive Learning for Cross-Domain Hyperspectral Image Representation
Lee, H., & Kwon, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747010 · Cited by 24 · see citing works
Learning to Align Sequential Actions in the Wild
Liu, W., Tekin, B., Coskun, H., Vineet, V., Fua, P., & Pollefeys, M. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00222 · Cited by 37 · see citing works
Duan, H., Zhao, N., Chen, K., & Lin, D.-H. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00301 · Cited by 28 · see citing works
Hierarchical Self-supervised Representation Learning for Movie Understanding
Xiao, F., Kundu, K., Tighe, J., & Modolo, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00950 · Cited by 23 · see citing works
Dave, I. R., Rizve, M. N., Chen, C., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00232 · Cited by 22 · see citing works
Controllable augmentations for video representation learning
Qian, R., Lin, W., See, J., & Li, D. (2024) · Visual Intelligence · doi:10.1007/s44267-023-00034-7 · Cited by 23 · see citing works
LAC - Latent Action Composition for Skeleton-based Action Segmentation
Yang, D., Wang, Y., Dantcheva, A., Kong, Q., Garattoni, L., Francesca, G., & Brémond, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01258 · Cited by 22 · see citing works
Dual Contrastive Learning for Spatio-temporal Representation
Ding, S., Qian, R., & Xiong, H. (2022) · ACM Multimedia · doi:10.1145/3503161.3547783 · Cited by 26 · see citing works
Audio-Visual Contrastive Learning with Temporal Self-Supervision
Jenni, S., Black, A., & Collomosse, J. P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.25967 · Cited by 21 · see citing works
PointClustering: Unsupervised Point Cloud Pre-training using Transformation Invariance in Clustering
Long, F.-C., Yao, T., Qiu, Z.-F., Li, L., & Mei, T. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.02090 · Cited by 25 · see citing works
Long, B., Goodin, S., Kachergis, G., Marchman, V. A., Radwan, S. F., Sparks, R. Z., Xiang, V., Zhuang, C., Hsu, O., Newman, B., et al. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02206-1 · Cited by 20 · see citing works
COMEDIAN: Self-Supervised Learning and Knowledge Distillation for Action Spotting Using Transformers
Denize, J., Liashuha, M., Rabarisoa, J., Orcesi, A., & Hérault, R. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00060 · Cited by 20 · see citing works
Scene Consistency Representation Learning for Video Scene Segmentation
Wu, H.-Q., Chen, K.-Y., Luo, Y., Qiao, R., Ren, B., Liu, H., Xie, W.-C., & Shen, L. (2022) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01363 · Cited by 25 · see citing works
BaSSL: Boundary-aware Self-Supervised Learning for Video Scene Segmentation
Mun, J., Shin, M., Han, G., Lee, S., Ha, S., Lee, J., & Kim, E. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_29 · Cited by 19 · see citing works
Koala: Key Frame-Conditioned Long Video-LLM
Tan, R., Sun, X., Hu, P., Wang, J.-H., Deilamsalehy, H., Plummer, B. A., Russell, B. C., & Saenko, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01289 · Cited by 19 · see citing works
How Severe Is Benchmark-Sensitivity in Video Self-supervised Learning?
Thoker, F. M., Doughty, H., Bagad, P., & Snoek, C. G. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_36 · Cited by 18 · see citing works
Zhao, W.-C., Zhou, W.-G., Hu, H.-Z., Wang, M., & Li, H.-Q. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3416881 · Cited by 20 · see citing works
Self-Supervised Scene-Debiasing for Video Representation Learning via Background Patching
Assefa, M., Jiang, W., Gedamu, K., Yilma, G., Kumeda, B., & Ayalew, M. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3193559 · Cited by 17 · see citing works
AssistQ: Affordance-Centric Question-Driven Task Completion for Egocentric Assistant
Wong, B., Chen, J., Wu, Y., Lei, S. W., Mao, D., Gao, D., & Shou, M. Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_28 · Cited by 17 · see citing works
Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations
Salehi, M., Gavves, E., Snoek, C. G. M., & Asano, Y. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01516 · Cited by 15 · see citing works
Asymmetric Masked Distillation for Pre-Training Small Foundation Models
Zhao, Z., Huang, B., Xing, S., Wu, G., Qiao, Y., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01752 · Cited by 15 · see citing works
Learning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 15 · see citing works
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 15 · see citing works
Motion Sensitive Contrastive Learning for Self-supervised Video Representation
Ni, J., Zhou, N., Qin, J., Wu, Q., Liu, J., Li, B., & Huang, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_27 · Cited by 14 · see citing works
Federated Self-supervised Learning for Video Understanding
Rehman, Y. A. U., Gao, Y., Shen, J., de Gusmão, P. P. B., & Lane, N. D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19821-2_29 · Cited by 14 · see citing works
AVForensics: Audio-driven Deepfake Video Detection with Masking Strategy in Self-supervision
Zhu, Y., Gao, J., & Zhou, X. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592218 · Cited by 14 · see citing works
Self-Supervised Representation Learning From Temporal Ordering of Automated Driving Sequences
Lang, C., Braun, A., Schillingmann, L., Haug, K., & Valada, A. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3354552 · Cited by 14 · see citing works
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Tseng, Y., Berry, L., Chen, Y.-T., Chiu, I.-H., Lin, H.-H., Liu, M., Peng, P., Shih, Y.-J., Wang, H., Wu, H., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445941 · Cited by 15 · see citing works
Conformer-Based Self-Supervised Learning For Non-Speech Audio Tasks
Srivastava, S., Wang, Y., Tjandra, A., Kumar, A., Liu, C., Singh, K., & Saraf, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746490 · Cited by 14 · see citing works
MaCLR: Motion-Aware Contrastive Learning of Representations for Videos
Xiao, F., Tighe, J., & Modolo, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_21 · Cited by 17 · see citing works
Anomaly-aware self-supervised feature learning for weakly supervised video anomaly detection
Yang, Z., Wang, G., Guo, Y., Bao, X., & Huang, D. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104379 · Cited by 13 · see citing works
Contrastive Spatio-Temporal Pretext Learning for Self-Supervised Video Representation
Zhang, Y., Po, L.-M., Xu, X., Liu, M., Wang, Y., Ou, W., Zhao, Y., & Yu, W.-Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20248 · Cited by 13 · see citing works
A Deep Autoencoder-Based Approach for Suspicious Action Recognition in Surveillance Videos
Ahmed, W., & Yousaf, M. H. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-08038-7 · Cited by 13 · see citing works
Learning Text-to-Video Retrieval from Image Captioning
Ventura, L., Schmid, C., & Varol, G. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02202-8 · Cited by 13 · see citing works
Wang, Z., Liu, C., Zhu, L., Wang, T., Zhang, S., & Dou, Q. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3532311 · Cited by 14 · see citing works
Qing, Z., Zhang, S., Huang, Z., Xu, Y., Wang, X., Tang, M., Gao, C., Jin, R., & Sang, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01345 · Cited by 12 · see citing works
Deng, A., Yang, T., & Chen, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01876 · Cited by 12 · see citing works
Video Foundation Models for Animal Behavior AnalysisPreprint
Sun, J. J., Zhou, H., Zhao, L., Yuan, L., Seybold, B., Hendon, D., Schroff, F., Ross, D. A., Adam, H., Hu, B., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.30.605655 · Cited by 12 · see citing works
Objects do not disappear: Video object detection by single-frame object location anticipation
Liu, X., Nejadasl, F. K., van Gemert, J. C., Booij, O., & Pintea, S. L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00640 · Cited by 11 · see citing works
Enhanced Motion-Text Alignment for Image-to-Video Transfer Learning
Zhang, W., Wan, C., Liu, T., Tian, X., Shen, X., & Ye, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01751 · Cited by 11 · see citing works
Hernandez, J., Villegas, R., & Ordóñez, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_25 · Cited by 11 · see citing works
Self-Supervised Video Representation Learning via Latent Time Navigation
Yang, D., Wang, Y., Kong, Q., Dantcheva, A., Garattoni, L., Francesca, G., & Brémond, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25416 · Cited by 11 · see citing works
Deep Unsupervised Representation Learning for Feature-Informed EEG Domain Extraction
Ng, H. W., & Guan, C. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3339179 · Cited by 11 · see citing works
- Authors
- Feichtenhofer, C., Fan, H., Xiong, B., Girshick, R., & He, K.
- Affiliations
- Meta (United States) (US)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 3298-3308
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3170837227
- Cited by
- 222
- References
- 104
- Citations by year
- 2026: 10 · 2025: 33 · 2024: 49 · 2023: 67 · 2022: 49 · 2021: 13 · 2020: 1
- FWCI
- 14.56
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- unsupervised feature learning, spatio-temporal representation learning, SlowFast, video understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
