The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Self-supervised Learning: Generative or Contrastive
Open accessAbstract
Deep supervised learning has achieved great success in the last decade. However, its defects of heavy dependence on manual labels and vulnerability to attacks have driven people to find other paradigms. As an alternative, self-supervised learning (SSL) attracts many researchers for its soaring performance on representation learning in the last several years. Self-supervised representation learning leverages input data itself as supervision and benefits almost all types of downstream tasks. In this survey, we take a look into new self-supervised learning methods for representation in computer vision, natural language processing, and graph learning. We comprehensively review the existing empirical methods and summarize them into three main categories according to their objectives: generative, contrastive, and generative-contrastive (adversarial). We further collect related theoretical analyses on self-supervised learning to provide deeper thoughts on why self-supervised learning works. Finally, we briefly discuss open problems and future directions for self-supervised learning. An outline slide for the survey is provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Self-citationUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
node2vec: Scalable Feature Learning for Networks
Grover, A., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939754 · Cited by 10,926
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Self-citationContext Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
LINE: Large-scale Information Network Embedding
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741093 · Cited by 5,416
Self-citationALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,178
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393
“Cloze Procedure”: A New Tool for Measuring Readability
Taylor, W. L. (1953) · Journalism Quarterly · doi:10.1177/107769905303000401 · Cited by 2,332
ArnetMiner: extraction and mining of academic social networks
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1402008 · Cited by 2,231
Self-citationGlobally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Conditional Image Generation with PixelCNN Decoders
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 1,833
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00300 · Cited by 1,827
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2019) · arXiv · doi:10.48550/arxiv.1907.10529 · Cited by 1,827
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., & Manning, C. D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1259 · Cited by 1,804 · cited 1×
“DEEP neural networks [71] have shown outstanding performance on various machine learning tasks, especially on supervised learning in computer vision (image classification [26], [48], [54], semantic segmentation [39], [76]), natural language processing (pre-trained language models [27], [68], [75], [138], sentiment ana…”
BackgroundHeterogeneous Graph Transformer
Hu, Z., Dong, Y., Wang, K., & Sun, Y. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380027 · Cited by 1,646
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591
NICE: Non-linear Independent Components Estimation
Dinh, L., Krueger, D., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1410.8516 · Cited by 1,493
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482
An End-to-End Deep Learning Architecture for Graph Classification
Zhang, M., Cui, Z., Neumann, M., & Chen, Y. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11782 · Cited by 1,461
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183
struc2vec: Learning Node Representations from Structural Identity
Ribeiro, L. F. R., Saverese, P. H. P., & Figueiredo, D. R. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098061 · Cited by 1,011
Learning Representations for Automatic Colorization
Larsson, G., Maire, M., & Shakhnarovich, G. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_35 · Cited by 946
Joint Unsupervised Learning of Deep Representations and Image Clusters
Yang, J., Parikh, D., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.556 · Cited by 879
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
Qiu, J., Chen, Q., Dong, Y., Zhang, J., Yang, H., Ding, M., Wang, K., & Tang, J. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403168 · Cited by 777
Self-citationGenerating diverse high-fidelity images with vq-vae-2
Razavi, A., Oord, A. V. D., & Vinyals, O. (2019) · arXiv · doi:10.48550/arxiv.1906.00446 · Cited by 709
An Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. (., & Wang, K. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2740908.2742839 · Cited by 658
GraphGAN: Graph Representation Learning With Generative Adversarial Nets
Wang, H., Wang, J., Wang, J., Zhao, M., Zhang, W., Zhang, F., Xie, X., & Guo, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11872 · Cited by 653
Self-citationSplit-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.76 · Cited by 616
DeepInf: Social Influence Prediction with Deep Learning
Qiu, J., Tang, J., Ma, H., Dong, Y., Wang, K., & Tang, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220077 · Cited by 587
Self-citationColorization as a Proxy Task for Visual Understanding
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.96 · Cited by 458
GPT-GNN: Generative Pre-Training of Graph Neural Networks
Hu, Z., Dong, Y., Wang, K., Chang, K.-W., & Sun, Y. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403237 · Cited by 446
Local Aggregation for Unsupervised Learning of Visual Embeddings
Zhuang, C., Zhai, A., & Yamins, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00610 · Cited by 417
Learning Structural Node Embeddings via Diffusion Wavelets
Donnat, C., Zitnik, M., Hallac, D., & Leskovec, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220025 · Cited by 360
SimGNN: A Neural Network Approach to Fast Graph Similarity Computation
Bai, Y., Ding, H., Bian, S., Chen, T., Sun, Y., & Wang, W. (2019) · Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining · doi:10.1145/3289600.3290967 · Cited by 325
Boosting Self-Supervised Learning via Knowledge Transfer
Noroozi, M., Vinjimoor, A., Favaro, P., & Pirsiavash, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00975 · Cited by 296
Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
Wang, Z., She, Q., & Ward, T. E. (2022) · ACM Computing Surveys · doi:10.1145/3439723 · Cited by 267
Self-citationAdversarial Representation Learning for Text-to-Image Matching
Sarafianos, N., Xu, X., & Kakadiaris, I. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00591 · Cited by 250
Sun, K., Lin, Z., & Zhu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6048 · Cited by 237
Cognitive Graph for Multi-Hop Reading Comprehension at Scale
Ding, M., Zhou, C., Chen, Q., Yang, H., & Tang, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1259 · Cited by 222
Self-citationDai, Q., Li, Q., Tang, J., & Wang, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11865 · Cited by 210
Self-citationOn Sampling Strategies for Neural Network-based Collaborative Filtering
Chen, T., Sun, Y., Shi, Y., & Hong, L. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098202 · Cited by 209
Triple Generative Adversarial Nets
Li, C., Xu, K., Zhu, J., & Zhang, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02291 · Cited by 204
Learning Classification with Unlabeled Data
de Sa, V. R. (1993) · Cited by 190
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Newell, A., & Deng, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00737 · Cited by 178
OAG: Toward Linking Large-scale Heterogeneous Entity Graphs
Zhang, F., Liu, X., Tang, J., Dong, Y., Yao, P., Zhang, J., Gu, X., Wang, Y., Shao, B., Li, R., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330785 · Cited by 175
Self-citationLearning Image Representations by Completing Damaged Jigsaw Puzzles
Kim, D., Cho, D., Yoo, D., & Kweon, I. S. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00092 · Cited by 172
ProNE: Fast and Scalable Network Representation Learning
Zhang, J., Dong, Y., Wang, Y., Tang, J., & Ding, M. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/594 · Cited by 171
Self-citationWei, C., Xie, L., Ren, X., Xia, Y., Su, C., Liu, J., Tian, Q., & Yuille, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00201 · Cited by 163
Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering
Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1911.10470 · Cited by 152
Domain Adaptation with Adversarial Training and Graph Embeddings
Alam, F., Joty, S., & Imran, M. A. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1099 · Cited by 148
Semi-supervised Learning on Graphs with Generative Adversarial Nets
Ding, M., Tang, J., & Zhang, J. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271768 · Cited by 144
Self-citationDeep Variational Network Embedding in Wasserstein Space
Zhu, D., Cui, P., Wang, D., & Zhu, W. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220052 · Cited by 142
Self-supervised Pretraining of Visual Features in the Wild
Goyal, P., Caron, M., Lefaudeux, B., Xu, M., Wang, P., Pai, V., Singh, M., Liptchinsky, V., Misra, I., Joulin, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.01988 · Cited by 141
Incorporating GAN for Negative Sampling in Knowledge Representation Learning
Wang, P., Li, S., & Pan, R. (2018) · AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11536 · Cited by 136
Unsupervised Visual Representation Learning by Graph-Based Consistent Constraints
Li, D., Hung, W.-C., Huang, J., Wang, S., Ahuja, N., & Yang, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_41 · Cited by 61
Research works printed but not linked: 22
- ganin, 2016, Domain-adversarial training of neural networks, Journal of Machine Learning Research, 17, 2096
- donahue, 2019, Large scale adversarial representation learning, Proc 33rd Int Conf Neural Inf Process Syst, 10541
- ho, 2019, Flow++: Improving flow-based generative models with variational dequantization and architecture design, Proc 36th Int Conf Mach Learn, 2722
- hu, 2019, Strategies for pre-training graph neural networks, Proc Int Conf Learn Representations
- zhu, 2017, Toward multimodal image-to-image translation, Proc 31st Int Conf Neural Inf Process Syst, 465
- goodfellow, 2014, Generative adversarial nets, Proc 27th Int Conf Neural Inf Process Syst, 2672
- arjovsky, 2017, Wasserstein generative adversarial networks, Proc 34th Int Conf Mach Learn, 214
- bachman, 2019, Learning representations by maximizing mutual information across views, Proc 33rd Int Conf Neural Inf Process Syst, 15509
- ballard, 1987, Modular learning in neural networks, Proc 6th Nat Conf Artif Intell, 279
- gutmann, 2010, Noise-contrastive estimation: A new estimation principle for unnormalized statistical models, Proc 13th Int Conf Artif Intell Statist, 297
- vaswani, 2017, Attention is all you need, Proc 31st Int Conf Neural Inf Process Syst, 5998
- van oord, 2016, Pixel recurrent neural networks, Proc 33rd Int Conf Mach Learn, 1747
- kingma, 2018, Glow: Generative flow with invertible 1x1 convolutions, Proc 32nd Int Conf Neural Inf Process Syst, 10215
- you, 2018, Graph convolutional policy network for goal-directed molecular graph generation, Proc 32nd Int Conf Neural Inf Process Syst, 6410
- you, 2018, GraphRNN: Generating realistic graphs with deep auto-regressive models, Proc 35th Int Conf Mach Learn, 5708
- krizhevsky, 2012, ImageNet classification with deep convolutional neural networks, Proc 25th Int Conf Neural Inf Process Syst, 1097
- shen, 2017, Style transfer from non-parallel text by cross-alignment, Proc 31st Int Conf Neural Inf Process Syst, 6830
- shen, 2017, Adversarial representation learning for domain adaptation, Stat, 1050
- sun, 2019, vGraph: A generative model for joint community detection and node representation learning, Proc 33rd Int Conf Neural Inf Process Syst, 512
- radford, 2019, Language models are unsupervised multitask learners, OpenAIRE blog, 1
- van den oord, 2016, WaveNet: A generative model for raw audio, Proc 9th ISCA Speech Synth Workshop, 125
- ng, 2011, Sparse autoencoder, Cs294a lecture notes, 72, 1
Cited by
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,025
Masked Autoencoders for Point Cloud Self-supervised Learning
Pang, Y., Wang, W., Tay, F. E. H., Liu, W., Tian, Y., & Yuan, L. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20086-1_35 · Cited by 605
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 552
Self-citation
Metadata
- Authors
- Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2021
- Pages
- 1-1
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W3035725276
- Cited by
- 1,087
- References
- 91
- FWCI
- 65.46
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- National Natural Science Foundation of China
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.