CoChat
Sign Up
Cited by 714Advanced Clustering Algorithms ResearchDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Contrastive Clustering

Open access
Li, Yunfan, Hu, Peng, Liu, Zitao, Peng, Dezhong, Zhou, Joey Tianyi, Peng, Xi·2021·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v35i10.17037
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we propose an online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature matrix could be regarded as soft labels of instances, and accordingly the columns could be further regarded as cluster representations. By simultaneously optimizing the instance- and cluster-level contrastive loss, the model jointly learns representations and cluster assignments in an end-to-end manner. Besides, the proposed method could timely compute the cluster assignment for each individual, even when the data is presented in streams. Extensive experimental results show that CC remarkably outperforms 17 competitive clustering methods on six challenging image benchmarks. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline. The code is available at https://github.com/XLearning-SCU/2021-AAAI-CC.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aaai.v35i10.17037

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Introducing clustering constraints into existing contrastive learning frameworks is an alternative, such as CC, PCL, and SwAV.

    “Therefore, introducing clustering constraints into existing contrastive learning frameworks is an alternative, such as CC [134], PCL [135], and SwAV [36].”

    From Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects (2024) · cites this paper as [134]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. Asano, Y. M.; Rupprecht, C.; and Vedaldi, A. 2019. Self- labelling via simultaneous clustering and representation learning. arXiv preprint arXiv:1911.05371 . · arXiv
  2. Cai, D.; He, X.; Wang, X.; Bao, H.; and Han, J. 2009. Lo- cality preserving nonnegative matrix factorization. In IJCAI, volume 9, 1010-1015.
  3. Caron, M.; Bojanowski, P.; Joulin, A.; and Douze, M. 2018. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), 132-149.
  4. Chang, J.; Guo, Y.; Wang, L.; Meng, G.; Xiang, S.; and Pan, C. 2019. Deep Discriminative Clustering Analysis. arXiv preprint arXiv:1905.01681 . · arXiv
  5. Chang, J.; Wang, L.; Meng, G.; Xiang, S.; and Pan, C. 2017a. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, 5879-5887.
  6. Chang, J.; Wang, L.; Meng, G.; Xiang, S.; and Pan, C. 2017b. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, 5879-5887.
  7. Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020. A simple framework for contrastive learning of visual repre- sentations. arXiv preprint arXiv:2002.05709 . · arXiv
  8. Coates, A.; Ng, A.; and Lee, H. 2011. An analysis of single- layer networks in unsupervised feature learning. In Proceed- ings of the fourteenth international conference on artificial intelligence and statistics, 215-223.
  9. Darlow, L. N.; and Storkey, A. 2020. DHOG: Deep Hier- archical Object Grouping. arXiv preprint arXiv:2003.08821 . · arXiv
  10. Dosovitskiy, A.; Springenberg, J. T.; Riedmiller, M.; and Brox, T. 2014. Discriminative unsupervised feature learning with convolutional neural networks. In Advances in neural information processing systems, 766-774.
  11. Ghasedi Dizaji, K.; Herandi, A.; Deng, C.; Cai, W.; and Huang, H. 2017. Deep clustering via joint convolutional au- toencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference on com- puter vision, 5736-5745.
  12. Gowda, K. C.; and Krishna, G. 1978. Agglomerative clus- tering using the concept of mutual nearest neighbourhood. Pattern recognition 10(2): 105-112.
  13. Grill, J.-B.; Strub, F.; Altché, F.; Tallec, C.; Richemond, P. H.; Buchatskaya, E.; Doersch, C.; Pires, B. A.; Guo, Z. D.; Azar, M. G.; et al. 2020. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733 . · arXiv
  14. Guo, X.; Gao, L.; Liu, X.; and Yin, J. 2017a. Improved deep embedded clustering with local structure preservation. In IJCAI, 1753-1759.
  15. Guo, X.; Liu, X.; Zhu, E.; and Yin, J. 2017b. Deep clustering with convolutional autoencoders. In International confer- ence on neural information processing, 373-382. Springer.
  16. Gutmann, M.; and Hyvärinen, A. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth Inter- national Conference on Artificial Intelligence and Statistics, 297-304.
  17. Hadsell, R.; Chopra, S.; and LeCun, Y. 2006. Dimension- ality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), volume 2, 1735-1742. IEEE.
  18. Haeusser, P.; Plapp, J.; Golkov, V.; Aljalbout, E.; and Cre- mers, D. 2018. Associative deep clustering: Training a clas- sification network with no labels. In German Conference on Pattern Recognition, 18-32. Springer.
  19. He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9729-9738.
  20. Hu, W.; Miyato, T.; Tokui, S.; Matsumoto, E.; and Sugiyama, M. 2017. Learning discrete representations via information maximizing self-augmented training. arXiv preprint arXiv:1702.08720 . · arXiv
  21. Huang, J.; Gong, S.; and Zhu, X. 2020. Deep Semantic Clus- tering by Partition Confidence Maximisation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
  22. Ji, X.; Henriques, J. F.; and Vedaldi, A. 2019. Invariant in- formation clustering for unsupervised image classification and segmentation. In Proceedings of the IEEE International Conference on Computer Vision, 9865-9874.
  23. Krizhevsky, A.; and Hinton, G. 2009. Learning multiple lay- ers of features from tiny images. Master's thesis, Depart- ment of Computer Science, University of Toronto .
  24. Li, J.; Zhou, P.; Xiong, C.; Socher, R.; and Hoi, S. C. 2020. Prototypical Contrastive Learning of Unsupervised Repre- sentations. arXiv preprint arXiv:2005.04966 . · arXiv
  25. Li, X.; Zhang, R.; Wang, Q.; and Zhang, H. 2020. Au- toencoder Constrained Clustering With Adaptive Neighbors. IEEE Transactions on Neural Networks and Learning Sys- tems 1-7.
  26. Liu, W.; Shen, X.; and Tsang, I. 2017. Sparse Embedded k- Means Clustering. In Advances in Neural Information Pro- cessing Systems, 3319-3327.
  27. Liu, X.; Dou, Y.; Yin, J.; Wang, L.; and Zhu, E. 2016. Mul- tiple kernel k-means clustering with matrix-induced regular- ization. In Proceedings of the thirtieth AAAI conference on artificial intelligence, 1888-1894.
  28. MacQueen, J.; et al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, 281-297.
  29. Nie, F.; Wang, X.; Jordan, M. I.; and Huang, H. 2016. The constrained laplacian rank algorithm for graph-based clus- tering. In AAAI, 1969-1976.
  30. Nie, F.; Zeng, Z.; Tsang, I. W.; Xu, D.; and Zhang, C. 2011. Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering. IEEE Transactions on Neural Networks 22(11): 1796-1808.
  31. Peng, X.; Feng, J.; Xiao, S.; Yau, W. Y.; Zhou, J. T.; and Yang, S. 2018. Structured AutoEncoders for Subspace Clus- tering. IEEE Trans Image Process 27(10): 5076-5086. ISSN 1057-7149. doi:10.1109/TIP.2018.2848470. · DOI
  32. Peng, X.; Xiao, S.; Feng, J.; Yau, W.; and Yi, Z. 2016. Deep Subspace Clustering with Sparsity Prior. In Proc. of 25th Int. Joint Conf. Artif. Intell., 1925-1931. New York, NY, USA.
  33. Peng, X.; Yi, Z.; and Tang, H. 2015. Robust subspace clustering via thresholding ridge regression. In AAAI, vol- ume 25, 3827-3833.
  34. Radford, A.; Metz, L.; and Chintala, S. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 . · arXiv
  35. Sharma, V.; Tapaswi, M.; Sarfraz, M. S.; and Stiefelhagen, R. 2020. Clustering based Contrastive Learning for Improv- ing Face Representations. arXiv preprint arXiv:2004.02195 . · arXiv
  36. Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.-A.; and Bottou, L. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning re- search 11(12).
  37. Wang, Z.; Li, Z.; Wang, R.; Nie, F.; and Li, X. 2020. Large graph clustering with simultaneous spectral embedding and discretization. IEEE Transactions on Pattern Analysis and Machine Intelligence .
  38. Wu, J.; Long, K.; Wang, F.; Qian, C.; Li, C.; Lin, Z.; and Zha, H. 2019. Deep comprehensive correlation mining for image clustering. In Proceedings of the IEEE International Conference on Computer Vision, 8150-8159.
  39. Xie, J.; Girshick, R.; and Farhadi, A. 2016. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, 478-487.
  40. Yang, J.; Parikh, D.; and Batra, D. 2016. Joint unsupervised learning of deep representations and image clusters. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5147-5156.
  41. Zeiler, M. D.; Krishnan, D.; Taylor, G. W.; and Fergus, R. 2010. Deconvolutional networks. In 2010 IEEE Computer Society Conference on computer vision and pattern recogni- tion, 2528-2535. IEEE.
  42. Zelnik-Manor, L.; and Perona, P. 2005. Self-tuning spectral clustering. In Advances in neural information processing systems, 1601-1608.
  43. Zhong, H.; Chen, C.; Jin, Z.; and Hua, X.-S. 2020. Deep robust clustering by contrastive learning. arXiv preprint arXiv:2008.03030 . · arXiv

Cited by

Sorted by citations

Metadata

Authors
Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J. T., & Peng, X.
Affiliations
  • Sichuan University (CN)
  • Southwest University (CN)
  • Agency for Science, Technology and Research (SG)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2021-05-18
Volume
35
Issue
10
Pages
8547-8555
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v35i10.17037
OpenAlex
W3087124270
Cited by
714
References
4
Citations by year
2026: 73 · 2025: 144 · 2024: 176 · 2023: 185 · 2022: 103 · 2021: 33
FWCI
56.92
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Sichuan University
  • Beijing Nova Program
  • Fundamental Research Funds for the Central Universities (YJ201949)
  • Northwest Fisheries Science Center (61625204, 61836006)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Clustering Algorithms Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
contrastive clustering, online clustering, instance-level contrastive learning, data augmentation, soft labels, cluster assignment, data streams, image clustering

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review