Unsupervised Learning of Visual Representations Using Videos
Open accessAbstract
Is strong supervision necessary for learning a good visual representation? Do we really need millions of semantically-labeled images to train a Convolutional Neural Network (CNN)? In this paper, we present a simple yet surprisingly powerful approach for unsupervised learning of CNN. Specifically, we use hundreds of thousands of unlabeled videos from the web to learn visual representations. Our key idea is that visual tracking provides the supervision. That is, two patches connected by a track should have similar visual representation in deep feature space since they probably belong to same object or object part. We design a Siamese-triplet network with a ranking loss function to train this CNN representation. Without using a single image from ImageNet, just using 100K unlabeled videos and the VOC 2012 dataset, we train an ensemble of unsupervised networks that achieves 52% mAP (no bounding box regression). This performance comes tantalizingly close to its ImageNet-supervised counterpart, an ensemble which achieves a mAP of 54.4%. We also show that our unsupervised network can perform competitively in other tasks such as surface-normal estimation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most deepfake detection methods assume a static game.
“Our survey found that most deepfake detection methods assume a static game (Mirsky and Lee 2021; Wang and Gupta 2015).”
From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Wang and Gupta 2015
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
SURF: Speeded Up Robust Features
Bay, H., Tuytelaars, T., & Van Gool, L. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_32 · Cited by 14,955 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · see citing works
High-Speed Tracking with Kernelized Correlation Filters
Henriques, J. F., Caseiro, R., Martins, P., & Batista, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2345390 · Cited by 5,753 · see citing works
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336 · see citing works
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · see citing works
Learning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · see citing works
(1989) · neural information processing systems · Cited by 3,736 · see citing works
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · see citing works
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Self-citationLee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412 · see citing works
Deep Metric Learning Using Triplet Network
Hoffer, E., & Ailon, N. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24261-3_7 · Cited by 2,134 · see citing works
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861 · see citing works
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · see citing works
Quattoni, A., & Torralba, A. B. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206537 · Cited by 1,533 · see citing works
Slow Feature Analysis: Unsupervised Learning of Invariances
Wiskott, L., & B, T. J. S. A. (2002) · Neural Computation · doi:10.1162/089976602317318938 · Cited by 1,351 · see citing works
Learning Fine-Grained Image Similarity with Deep Ranking
Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J. F., Chen, B., & Wu, Y. N. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.180 · Cited by 1,282 · see citing works
Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
Ranzato, M., Huang, F., Boureau, Y.-L., & LeCun, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383157 · Cited by 1,186 · see citing works
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070 · see citing works
Multimodal Learning with Deep Boltzmann Machines
Srivastava, N., & Salakhutdinov, R. (2012) · doi:10.5555/2627435.2697059 · Cited by 1,070 · see citing works
Discovering objects and their location in images
Šivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.77 · Cited by 973 · see citing works
Le, Q. V., Zou, W. Y., Yeung, S. Y., & Ng, A. Y. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995496 · Cited by 966 · see citing works
Pedestrian Detection with Unsupervised Multi-stage Feature Learning
Sermanet, P., Kavukcuoglu, K., Chintala, S., & LeCun, Y. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.465 · Cited by 753 · see citing works
Learning invariance from transformation sequences
Földiák, P. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.194 · Cited by 661 · cited 1× · see citing works
“Therefore, researchers have started focusing on learning feature representations using videos [11, 53, 27, 43, 56, 16, 48, 34, 45].”
BackgroundConvolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_11 · Cited by 645 · see citing works
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 633 · see citing works
Using Multiple Segmentations to Discover Objects and their Extent in Image Collections
Russell, B. C., Freeman, W. T., Efros, A. A., Šivic, J., & Zisserman, A. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.326 · Cited by 612 · see citing works
Unsupervised Discovery of Mid-Level Discriminative Patches
Singh, S., Gupta, A., & Efros, A. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_6 · Cited by 517 · see citing works
Self-citationZhang, R., Lin, L., Zhang, R., Zuo, W., & Lei Zhang, A. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2467315 · Cited by 504 · see citing works
Learning descriptors for object recognition and 3D pose estimation
Wohlhart, P., & Lepetit, V. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298930 · Cited by 452 · see citing works
Designing deep networks for surface normal estimation
Wang, X., Fouhey, D. F., & Gupta, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298652 · Cited by 343 · see citing works
Self-citationDeep learning from temporal coherence in video
Mobahi, H., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553469 · Cited by 328 · see citing works
Deep Convolutional Ranking for Multilabel Image AnnotationPreprint
Gong, Y., Jia, Y., Leung, T., Toshev, A., & Ioffe, S. (2014) · Cited by 272 · see citing works
Discriminative Decorrelation for Clustering and Classification
Hariharan, B., Malik, J., & Ramanan, D. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_33 · Cited by 271 · see citing works
Mid-level Visual Element Discovery as Discriminative Mode Seeking
Doersch, C., Gupta, A., & Efros, A. A. (2013) · neural information processing systems · Cited by 238 · see citing works
Self-citationHierarchical face parsing via deep learning
Luo, P., Wang, X., & Tang, X. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247963 · Cited by 190 · see citing works
Self-citationRobust Boltzmann Machines for recognition and denoising
Tang, Y., Salakhutdinov, R., & Hinton, G. E. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247936 · Cited by 171 · see citing works
Data-Driven 3D Primitives for Single Image Understanding
Fouhey, D. F., Gupta, A., & Hebert, M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.421 · Cited by 167 · see citing works
Self-citationUnsupervised Learning of Spatiotemporally Coherent Metrics
Goroshin, R., Bruna, J., Tompson, J., Eigen, D., & LeCun, Y. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.465 · Cited by 156 · see citing works
Matching-CNN meets KNN: Quasi-parametric human parsing
Liu, S., Liang, X., Liu, L., Shen, X., Yang, J., Xu, C., Lin, L., Cao, X., & Yan, S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298748 · Cited by 142 · see citing works
Deep Learning of Invariant Features via Simulated Fixations in Video
Zou, W. Y., Shenghuo Zhu, B., Yu, K., & Ng, A. Y. (2012) · Cited by 131 · see citing works
Describing Visual Scenes using Transformed Dirichlet Processes
Torralba, A. B., Willsky, A. S., Sudderth, E. B., & Freeman, W. T. (2005) · neural information processing systems · Cited by 122 · see citing works
Discriminatively Trained Dense Surface Normal Estimation
Ladický, Ľ., Zeisl, B., & Pollefeys, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_31 · Cited by 120 · see citing works
Unfolding an Indoor Origami World
Fouhey, D. F., Gupta, A., & Hebert, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_44 · Cited by 85 · see citing works
Self-citationThe Shape Boltzmann Machine: A strong model of object shape
Eslami, S. M. A., Heess, N., & Winn, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247702 · Cited by 61 · see citing works
Context as Supervisory Signal: Discovering Objects with Predictable Context
Doersch, C., Gupta, A., & Efros, A. A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_24 · Cited by 39 · see citing works
Self-citationUnsupervised learning of invariant features using video
Stavens, D., & Thrun, S. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539773 · Cited by 36 · see citing works
Computational Baby LearningPreprint
Liang, X., Liu, S., Wei, Y., Liu, L., Lin, L., & Yan, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2861 · Cited by 17 · see citing works
Flagged first, then by citations
1 of the 58 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
“In Computer Vision [28] use a triplet loss on tracked video patches so that patches from the same object at different timesteps are more similar to each other than to random patches.”
BackgroundMethodContext Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,803 · see citing works
Unsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
Training Region-Based Object Detectors with Online Hard Example Mining
Shrivastava, A., Gupta, A., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.89 · Cited by 2,686 · see citing works
Self-citationUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Self-citationDeep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674 · see citing works
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
Sun, C., Shrivastava, A., Singh, S., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.97 · Cited by 2,484 · see citing works
Self-citationSelf-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · see citing works
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997 · see citing works
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
Clark, K., Luong, M.-T., Le, Q. V., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.10555 · Cited by 1,844 · see citing works
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
Huang, L., Zhao, X., & Huang, K. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2957464 · Cited by 1,809 · see citing works
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · see citing works
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435 · see citing works
Network Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389 · see citing works
Adversarial Feature LearningPreprint
Donahue, J., Krähenbühl, P., & Darrell, T. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1605.09782 · Cited by 1,982 · see citing works
Rethinking ImageNet Pre-Training
He, K., Girshick, R., & Dollár, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00502 · Cited by 1,164 · see citing works
Taskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128 · see citing works
Data-Efficient Image Recognition with Contrastive Predictive Coding
Hénaff, O. J., Srinivas, A., De Fauw, J., Razavi, A., Doersch, C., Eslami, S. M. A., & Oord, A. V. D. (2019) · arXiv · doi:10.48550/arxiv.1905.09272 · Cited by 1,014 · see citing works
Generating Videos with Scene DynamicsPreprint
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.02612 · Cited by 976 · see citing works
Contrastive Representation Learning: A Framework and Review
Le-Khac, P. H., Healy, G., & Smeaton, A. F. (2020) · IEEE Access · doi:10.1109/access.2020.3031549 · Cited by 862 · see citing works
Domain Generalization by Solving Jigsaw Puzzles
Carlucci, F. M., D’Innocente, A., Bucci, S., Barbara Caputo, B., & Tommasi, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00233 · Cited by 838 · see citing works
Arandjelović, R., & Zisserman, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 836 · see citing works
Joint Unsupervised Learning of Deep Representations and Image Clusters
Yang, J., Parikh, D., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.556 · Cited by 879 · see citing works
Human Action Recognition and Prediction: A Survey
Kong, Y., & Fu, Y. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01594-9 · Cited by 832 · see citing works
Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification
Misra, I., Zitnick, C. L., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_32 · Cited by 779 · see citing works
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 766 · see citing works
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704 · see citing works
Deep Predictive Coding Networks for Video Prediction and Unsupervised LearningPreprint
Lotter, W. E., Kreiman, G., & Cox, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08104 · Cited by 805 · see citing works
A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection
Wang, X., Shrivastava, A., & Gupta, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.324 · Cited by 655 · see citing works
Self-citationGoing deeper into action recognition: A survey
Herath, S., Harandi, M. T., & Porikli, F. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.010 · Cited by 646 · see citing works
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.76 · Cited by 724 · see citing works
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
Ye, M., Zhang, X., Yuen, P. C., & Chang, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00637 · Cited by 629 · see citing works
End-to-End Learning of Visual Representations From Uncurated Instructional Videos
Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Šivic, J., & Zisserman, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00990 · Cited by 622 · see citing works
Multi-task Self-Supervised Visual Learning
Doersch, C., & Zisserman, A. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.226 · Cited by 682 · see citing works
Generative Image Modeling Using Style and Structure Adversarial Networks
Wang, X., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_20 · Cited by 607 · see citing works
Self-citationSelf-supervised learning for medical image analysis using image context restoration
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101539 · Cited by 575 · see citing works
Time-Contrastive Networks: Self-Supervised Learning from Video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Stefan Schaal, A., Levine, S., & Brain, G. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462891 · Cited by 579 · see citing works
A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02443-6 · Cited by 704 · see citing works
Few-Shot Image Recognition by Predicting Parameters from Activations
Qiao, S., Liu, C., Shen, W., & Yuille, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00755 · Cited by 565 · see citing works
RankIQA: Learning from Rankings for No-Reference Image Quality Assessment
Liu, X., van de Weijer, J., & Bagdanov, A. D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.118 · Cited by 543 · see citing works
Unsupervised Representation Learning by Sorting Sequences
Lee, H.-Y., Huang, J., Singh, M., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.79 · Cited by 529 · see citing works
Anticipating Visual Representations from Unlabeled Video
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.18 · Cited by 538 · see citing works
CURL: Contrastive Unsupervised Representations for Reinforcement LearningPreprint
Srinivas, A., Laskin, M., & Abbeel, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.04136 · Cited by 521 · see citing works
Colorization as a Proxy Task for Visual Understanding
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.96 · Cited by 527 · see citing works
Learning from Noisy Large-Scale Datasets with Minimal Supervision
Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., & Belongie, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.696 · Cited by 480 · see citing works
Self-citationAn Uncertain Future: Forecasting from Static Images Using Variational Autoencoders
Walker, J., Doersch, C., Gupta, A., & Hebert, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_51 · Cited by 467 · see citing works
Self-citationWhat makes ImageNet good for transfer learning?Preprint
Huh, M., Agrawal, P., & Efros, A. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.08614 · Cited by 591 · see citing works
Learning Correspondence From the Cycle-Consistency of Time
Wang, X., Jabri, A., & Efros, A. A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00267 · Cited by 452 · see citing works
Self-citationSpatiotemporal Contrastive Video Representation Learning
Qian, R., Meng, T., Gong, B., Yang, M., Wang, H., Belongie, S., & Cui, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00689 · Cited by 462 · see citing works
Self-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421 · see citing works
Learning Features by Watching Objects Move
Pathak, D., Girshick, R. B., Dollár, P., Darrell, T., & Hariharan, B. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.638 · Cited by 549 · see citing works
Arandjelović, R., & Zisserman, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_27 · Cited by 448 · see citing works
Wang, N., Song, Y., Ma, C., Zhou, W., Liu, W., & Li, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00140 · Cited by 443 · see citing works
Proxy Anchor Loss for Deep Metric Learning
Kim, S., Kim, D.-W., Cho, M., & Kwak, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00330 · Cited by 437 · see citing works
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909 · see citing works
Zhao, H., Gan, C., Rouditchenko, A., Vondrick, C., McDermott, J. H., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_35 · Cited by 435 · see citing works
Local Aggregation for Unsupervised Learning of Visual Embeddings
Zhuang, C., Zhai, A., & Yamins, D. L. K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00610 · Cited by 422 · see citing works
Temporal Generative Adversarial Nets with Singular Value Clipping
Saito, M., Matsumoto, E., & Saito, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.308 · Cited by 412 · see citing works
With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00945 · Cited by 404 · see citing works
Tracking Emerges by Colorizing Videos
Vondrick, C., Shrivastava, A., Fathi, A., Guadarrama, S., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_24 · Cited by 393 · see citing works
Ambient Sound Provides Supervision for Visual Learning
Owens, A., Wu, J., McDermott, J. H., Freeman, W. T., & Torralba, A. B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_48 · Cited by 385 · see citing works
Video Representation Learning by Dense Predictive Coding
Han, T., Xie, W., & Zisserman, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00186 · Cited by 383 · see citing works
Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387 · see citing works
Generating Videos with Scene Dynamics
Vondrick, C., Pirsiavash, H., & Torralba, A. (2016) · Maryland Shared Open Access Repository · doi:10.13016/m26gih-tnyz · Cited by 441 · see citing works
Imbalanced Deep Learning by Minority Class Incremental Rectification
Dong, Q., Gong, S., & Zhu, X. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2832629 · Cited by 370 · see citing works
An Unsupervised Autoregressive Model for Speech Representation Learning
Chung, Y.-A., Hsu, W.-N., Tang, H., & Glass, J. R. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1473 · Cited by 356 · see citing works
Owens, A., Isola, P., McDermott, J. H., Torralba, A. B., Adelson, E. H., & Freeman, W. T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.264 · Cited by 352 · see citing works
Scaling and Benchmarking Self-Supervised Visual Representation Learning
Goyal, P., Mahajan, D. K., Gupta, A., & Misra, I. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00649 · Cited by 344 · see citing works
Self-citationDual Motion GAN for Future-Flow Embedded Video Prediction
Liang, X., Lee, L., Dai, W., & Xing, E. P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.194 · Cited by 342 · see citing works
Representation Learning by Learning to Count
Noroozi, M., Pirsiavash, H., & Favaro, P. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.628 · Cited by 379 · see citing works
You, C., Zhou, Y., Zhao, R., Staib, L. H., & Duncan, J. S. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3161829 · Cited by 333 · see citing works
Interpreting Deep Visual Representations via Network Dissection
Zhou, B., Bau, D., Oliva, A., & Torralba, A. B. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858759 · Cited by 334 · see citing works
SfM-Net: Learning of Structure and Motion from VideoPreprint
Vijayanarasimhan, S., Ricco, S., Schmid, C., Sukthankar, R., & Fragkiadaki, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07804 · Cited by 327 · see citing works
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Kim, D., Cho, D., & Kweon, I. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018545 · Cited by 314 · see citing works
Hard-Aware Deeply Cascaded Embedding
Yuan, Y., Yang, K., & Zhang, C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.94 · Cited by 313 · see citing works
Unsupervised deep learning for optical flow estimation
Ren, Z., Yan, J., Ni, B., Liu, B., Yang, X., & Zha, H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10723 · Cited by 296 · see citing works
On Mutual Information Maximization for Representation LearningPreprint
Tschannen, M., Djolonga, J., Rubenstein, P. K., Gelly, S., & Lučić, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.13625 · Cited by 340 · see citing works
A Review on Deep Learning Techniques for Video Prediction
Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., Garcia-Rodriguez, J., & Argyros, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045007 · Cited by 301 · see citing works
Boosting Self-Supervised Learning via Knowledge Transfer
Noroozi, M., Vinjimoor, A., Favaro, P., & Pirsiavash, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00975 · Cited by 298 · see citing works
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298 · see citing works
Localizing and Orienting Street Views Using Overhead Imagery
Vo, N., & Hays, J. H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_30 · Cited by 297 · see citing works
Learning to Learn: Model Regression Networks for Easy Small Sample Learning
Wang, Y.-X., & Hebert, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_37 · Cited by 296 · see citing works
Deep Metric Learning via Facility Location
Song, H. O., Jegelka, S., Rathod, V., & Murphy, K. P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.237 · Cited by 293 · see citing works
Geography-Aware Self-Supervised Learning
Ayush, K., Uzkent, B., Meng, C., Tanmay, K., Burke, M., Lobell, D., & Ermon, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01002 · Cited by 288 · see citing works
Self-Supervised Pretraining of 3D Features on any Point-Cloud
Zhang, Z., Girdhar, R., Joulin, A., & Misra, I. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01009 · Cited by 285 · see citing works
Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
Chen, Y., Pont-Tuset, J., Montes, A., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00130 · Cited by 273 · see citing works
Unsupervised pre-trained filter learning approach for efficient convolution neural network
Rehman, S. U., Tu, S., Waqas, M. M., Huang, Y., Rehman, O. U., Ahmad, B., & Ahmad, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.084 · Cited by 271 · see citing works
Unsupervised Pre-Training of Image Features on Non-Curated Data
Caron, M., Bojanowski, P., Mairal, J., & Joulin, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00305 · Cited by 258 · see citing works
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised LearningPreprint
Sajjadi, M. S. M., Javanmardi, M., & Taşdizen, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04586 · Cited by 874 · see citing works
Ranked List Loss for Deep Metric Learning
Wang, X., Yang, H., Kodirov, E., Hu, G., Garnier, R., & Robertson, N. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00535 · Cited by 255 · see citing works
Self-citationMulti-object Tracking with Quadruplet Convolutional Neural Networks
Son, J., Baek, M., Cho, M., & Han, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.403 · Cited by 245 · see citing works
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning
Wortsman, M., Ehsani, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00691 · Cited by 248 · see citing works
Unsupervised Visual Representation Learning by Context PredictionPreprint
Doersch, C., Gupta, A., & Efros, A. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05192 · Cited by 245 · see citing works
Self-citationSelf-Taught convolutional neural networks for short text clustering
Xu, J., Xu, B., Wang, P., Zheng, S., Tian, G., Zhao, J., & Xu, B. (2017) · Neural Networks · doi:10.1016/j.neunet.2016.12.008 · Cited by 243 · see citing works
- Authors
- Wang, X., & Gupta, A.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2794-2802
- OpenAlex
- W219040644
- Cited by
- 1,005
- References
- 58
- Citations by year
- 2026: 8 · 2025: 32 · 2024: 44 · 2023: 63 · 2022: 97 · 2021: 146 · 2020: 173 · 2019: 140 · 2018: 121 · 2017: 108 · 2016: 65 · 2015: 8
- FWCI
- 41.29
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1320083)
- Nvidia
- Defense Advanced Research Projects Agency (FA8750-14- 2-0244, FA8750)
- Multidisciplinary University Research Initiative
- Office of Naval Research (N00014-10-10934, MURI N000141010934)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- video-based learning, visual object tracking, ranking loss, object detection, unlabeled video, surface normal estimation, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review