The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding
Open accessAbstract
Manual annotation of large-scale point cloud dataset for varying tasks such as 3D object classification, segmentation and detection is often laborious owing to the irregular structure of point clouds. Self-supervised learning, which operates without any human labeling, is a promising approach to address this issue. We observe in the real world that humans are capable of mapping the visual concepts learnt from 2D images to understand the 3D world. Encouraged by this insight, we propose CrossPoint, a simple cross-modal contrastive learning approach to learn transferable 3D point cloud representations. It enables a 3D-2D correspondence of objects by maximizing agreement between point clouds and the corresponding rendered 2D image in the invariant space, while encouraging invariance to transformations in the point cloud modality. Our joint training objective combines the feature correspondences within and across modalities, thus ensembles a rich learning signal from both 3D point cloud and 2D image modalities in a self-supervised fashion. Experimental results show that our approach outperforms the previous unsupervised learning methods on a diverse range of downstream tasks including 3D object classification and segmentation. Further, the ablation studies validate the potency of our approach for a better point cloud understanding. Code and pretrained models are available at https://github.com/MohamedAfham/CrossPoint.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Dynamic Graph CNN for Learning on Point Clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019) · ACM Transactions on Graphics · doi:10.1145/3326362 · Cited by 7,199
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 2×
“Note that our approach doesn’t use any memory bank following the recent advancements in self-supervised contrastive learning [5, 13, 21].”
BackgroundMethodDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298801 · Cited by 4,550
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846
KPConv: Flexible and Deformable Convolution for Point Clouds
Thomas, H., Qi, C. R., Deschaud, J.-E., Marcotegui, B., Goulette, F., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00651 · Cited by 3,212
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02413 · Cited by 2,114
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393
FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation
Yang, Y., Feng, C., Shen, Y., & Tian, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00029 · Cited by 1,313
Deep Hough Voting for 3D Object Detection in Point Clouds
Qi, C. R., Litany, O., He, K., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00937 · Cited by 1,244
A scalable active framework for region annotation in 3D shape collections
Yi, L., Kim, V. G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., & Guibas, L. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980238 · Cited by 1,231
SO-Net: Self-Organizing Network for Point Cloud Analysis
Li, J., Chen, B. M., & Lee, G. H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00979 · Cited by 985
Relation-Shape Convolutional Neural Network for Point Cloud Analysis
Liu, Y., Fan, B., Xiang, S., & Pan, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00910 · Cited by 952
Rethinking Few-Shot Image Classification: A Good Embedding is All You Need?
Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J. B., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_16 · Cited by 952
Meta-learning with differentiable closed-form solversPreprint
Bertinetto, L., Henriques, J. F., Torr, P. H. S., & Vedaldi, A. (2018) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1805.08136 · Cited by 868
Uy, M. A., Pham, Q.-H., Hua, B.-S., Nguyen, T., & Yeung, S.-K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00167 · Cited by 842
Arandjelovic, R., & Zisserman, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.73 · Cited by 782
MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., & Carion, N. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00180 · Cited by 729
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
Xie, S., Gu, J., Guo, D., Qi, C. R., Guibas, L., & Litany, O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_34 · Cited by 609
PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds
Xu, M., Ding, R., Zhao, H., & Qi, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00319 · Cited by 523
An End-to-End Transformer Model for 3D Object Detection
Misra, I., Girdhar, R., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00290 · Cited by 499
Compositional Attention Networks for Machine ReasoningPreprint
Hudson, D. A., & Manning, C. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03067 · Cited by 491
Spatiotemporal Contrastive Video Representation Learning
Qian, R., Meng, T., Gong, B., Yang, M.-H., Wang, H., Belongie, S., & Cui, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00689 · Cited by 443
Zhao, Y., Birdal, T., Deng, H., & Tombari, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00110 · Cited by 351
VirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01101 · Cited by 342
Self-Supervised Pretraining of 3D Features on any Point-Cloud
Zhang, Z., Girdhar, R., Joulin, A., & Misra, I. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01009 · Cited by 278
Hard Negative Mixing for Contrastive Learning
Kalantidis, Y., Sarıyıldız, M. B., Pion, N., Weinzaepfel, P., & Larlus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01028 · Cited by 257
Multiresolution Tree Networks for 3D Point Cloud Processing
Gadelha, M., Wang, R., & Maji, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_7 · Cited by 242
Unsupervised Point Cloud Pre-training via Occlusion Completion
Wang, H., Liu, Q., Yue, X., Lasenby, J., & Kusner, M. J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00964 · Cited by 236
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
Xu, Q., Wang, W., Ceylan, D., Měch, R., & Neumann, U. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10711 · Cited by 234
DeLoache, J. S., Strauss, M. S., & Maynard, J. (1979) · Infant Behavior and Development · doi:10.1016/s0163-6383(79)80010-7 · Cited by 233
Audio-Visual Instance Discrimination with Cross-Modal Agreement
Morgado, P., Vasconcelos, N., & Misra, I. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01229 · Cited by 211
Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds
Huang, S., Xie, Y., Zhu, S.-C., & Zhu, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00647 · Cited by 193
H3DNet: 3D Object Detection Using Hybrid Geometric Primitives
Zhang, Z., Sun, B., Yang, H., & Huang, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_19 · Cited by 188
Unsupervised Multi-Task Feature Learning on Point Clouds
Hassani, K., & Haley, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00825 · Cited by 182
Han, Z.-Z., Wang, X., Liu, Y.-S., & Zwicker, M. (2019) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2019.01054 · Cited by 159
Self-Supervised Learning of Point Clouds via Orientation Estimation
Poursaeed, O., Jiang, T., Qiao, H., Xu, N., & Kim, V. G. (2020) · International Conference on 3D Vision · doi:10.1109/3dv50981.2020.00112 · Cited by 145
Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds
Rao, Y., Lu, J., & Zhou, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00542 · Cited by 135
Han, Z., Shang, M., Liu, Y.-S., & Zwicker, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018376 · Cited by 130
Self-Supervised Few-Shot Learning on Point CloudsPreprint
Sharma, C., & Kaul, M. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2009.14168 · Cited by 129
Learning Visual Representations with Caption Annotations
Sarıyıldız, M. B., Pérez, J., & Larlus, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_10 · Cited by 128
Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning
Du, B., Gao, X., Hu, W., & Li, X. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475458 · Cited by 107
Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D PretrainingPreprint
Liu, Y.-C., Huang, Y., Chiang, H.-Y., Su, H.-T., Liu, Z., Chen, C.-T., Tseng, C.-Y., & Hsu, W. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.04687 · Cited by 97
Sanghi, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_37 · Cited by 95
Zhang, L., & Zhu, Z. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00051 · Cited by 92
Improving Contrastive Learning by Visualizing Feature Transformation
Zhu, R., Zhao, B., Liu, J., Sun, Z., & Chen, C. W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01014 · Cited by 80
Self-supervised Video Representation Learning by Context and Motion Decoupling
Huang, L., Yu, L., Wang, B., Pan, P., Xu, Y., & Jin, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01367 · Cited by 76
Infants' Transfer of Response between Two-Dimensional and Three-Dimensional Stimuli
Rose, S. A. (1977) · Child Development · doi:10.2307/1128366 · Cited by 62
Self-Supervised Learning on 3D Point Clouds by Learning Discrete Generative Models
Eckart, B., Yuan, W., Liu, C., & Kautz, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00815 · Cited by 58
Progressive Seed Generation Auto-encoder for Unsupervised Point Cloud Learning
Yang, J., Ahn, P., Kim, D., Lee, H., & Kim, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00635 · Cited by 25
Interpretable Visual Reasoning via Induced Symbolic Space
Wang, Z., Wang, K., Yu, M., Xiong, J., Hwu, W., Hasegawa‐Johnson, M., & Shi, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00189 · Cited by 19
Self-supervised Temporal Discriminative Learning for Video Representation LearningPreprint
Wang, J., Lin, Y., Ma, A. J., & Yuen, P. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02129 · Cited by 15
Image2Point: 3D Point-Cloud Understanding with Pretrained 2D ConvNets.Preprint
Xu, C., Yang, S., Zhai, B., Wu, B., Yue, X., Zhan, W., Vajda, P., Keutzer, K., & Tomizuka, M. (2021) · arXiv (Cornell University) · Cited by 14
Point Cloud Pre-training by Mixing and DisentanglingPreprint
Sun, C., Zheng, Z., Wang, X., Xu, M., & Yang, Y. (2021) · arXiv (Cornell University) · Cited by 6
Research works printed but not linked: 8
- hengshuang, 0, Point transformer, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 16259
- xing, 2019, Adaptive cross-modal few-shot learning, Advances in neural information processing systems, 32
- liu, 2021, Point discriminative learning for unsupervised representation learning on 3D point clouds, ArXiv Preprint
- li, 2018, Pointcnn: Convolution on x-transformed points, Advances in neural information processing systems, 31
- afham, 0, Rich semantics improve few-shot learning, the 32nd British Machine Vision Conference
- achlioptas, 0, Learning representations and generative models for 3D point clouds, Proceedings of the 35th International Conference on Machine Learning, 80, 40
- xie, 0, PointContrast: Unsupervised pretraining for 3D point cloud understanding, European Conference on Computer Vision (ECCV)
- sauder, 2019, Self-supervised deep learning on point clouds by reconstructing space, Advances in neural information processing systems, 32, 12962
Cited by
CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIP
Zhang, J., Dong, R., & Ma, K. (2023) · 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw60793.2023.00219 · Cited by 62
ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding
Guo, Z., Tang, Y., Zhang, R., Wang, D., Wang, Z., Zhao, B., & Li, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01410 · Cited by 44
Unbiased Embodied Visual Representation Learning with Causal Inference and Cross-Modality Alignment
Kang, J., Chen, B., Zhong, P., Wang, Y., Yang, H., & Sheng, Y. (2025) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3760261
Metadata
- Authors
- Afham, M., Dissanayake, I., Dissanayake, D., Dharmasiri, A., Thilakarathna, K., & Rodrigo, R.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 9892-9902
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312317653
- Cited by
- 303
- References
- 65
- FWCI
- 53.93
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.