Multimodal supervised contrastive learning in remote sensing downstream tasks
Abstract
To leverage the large amount of unlabelled data available in remote sensing datasets, self-supervised learning (SSL) methods have recently emerged as an ubiquitous tool to pre-train robust image encoder models from unlabelled images. However, when used in a downstream setting, these models often need to be finetuned for a specific task after their pre-training. This finetuning still requires labelling information in order to train a classifier on top of the encoder while also updating the encoder weights. In this paper, we investigate the specific task of multimodal scene classification where a sample is composed of multiple views from multiple heterogeneous satellite sensors. We propose a method to improve the categorical cross-entropy finetuning process which is often used to specify the model for this downstream task. Our approach, based on the Supervised Contrastive Learning, uses the label information available to train an image encoder in a contrastive manner from multiple modalities while also training the task-specific classifier online. Such a multimodal supervised contrastive loss helps to better align representations from samples coming from multiple sensors but having the same class labels, thus improving the performance of the finetuning process. Our experiments on two public datasets including DFC2020 and Meter-ML with Sentinel-1/Sentinel-2 images show a significant gain over the baseline multimodal cross-entropy loss. All codes and datasets will be made publicly available for reproducibility upon acceptance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CLIP trains vision and language models with contrastive learning, aligning images and text in a shared space for consistency.
“The popular method CLIP [181] trains both vision and language models using contrastive learning, aligning images and text in a shared space for consistency.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [181]
CLIP uses Visual-Linguistic Contrast learning so that images and text share the same latent space.
“e.g., CLIP [181] employs Visual-Linguistic Contrast learning so that images and text share the same latent space.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [181]
CLIP and ALIGN use large-scale weakly annotated or natural language supervised learning to learn generalised visual language representations.
“models such as CLIP [181], ALIGN [182], etc., which use large-scale weakly annotated or natural language supervised learning of generalised visual language representations”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [181]
Cites
Supervised Contrastive Learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. (2020) · arXiv · doi:10.48550/arxiv.2004.11362 · Cited by 3,569 · see citing works
Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762307 · Cited by 3,400 · see citing works
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456 · see citing works
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · see citing works
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997 · see citing works
Artificial Intelligence for Remote Sensing Data Analysis: A review of challenges and opportunities
Zhang, L., & Zhang, L. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3145854 · Cited by 613 · see citing works
Self-Supervised Learning in Remote Sensing: A review
Wang, Y., Albrecht, C. M., Braham, N. A. A., Mou, L., & Zhu, X. X. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3198244 · Cited by 364 · see citing works
Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding
Stojnić, V., & Risojević, V. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00129 · Cited by 125 · see citing works
Self-supervised Vision Transformers for Land-cover Segmentation and Classification
Scheibenreif, L., Hanna, J., Mommert, M., & Borth, D. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00148 · Cited by 122 · see citing works
Berg, P., Pham, M., & Courty, N. (2022) · Remote Sensing · doi:10.3390/rs14163995 · Cited by 77 · see citing works
Self-Supervised Learning for Invariant Representations From Multi-Spectral and SAR Images
Jain, P., Schoen-Phelan, B., & Ross, R. J. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3204888 · Cited by 53 · see citing works
Change Detection Based on Supervised Contrastive Learning for High-Resolution Remote Sensing Imagery
Wang, J., Zhong, Y., & Zhang, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3236664 · Cited by 44 · see citing works
Cross-Modal Contrastive Learning for Remote Sensing Image Classification
Feng, Z., Song, L., Yang, S., Zhang, X., & Jiao, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3296703 · Cited by 40 · see citing works
On the duality between contrastive and non-contrastive self-supervised learningPreprint
Garrido, Q., Chen, Y., Bardes, A., Najman, L. A., & LeCun, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.02574 · Cited by 30 · see citing works
Schmitt, M., Hughes, L. H., Qiu, C., & Zhu, X. X. (2019) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-iv-2-w7-153-2019 · Cited by 28 · see citing works
Huang, M., Dong, L., Dong, W., & Shi, G. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3275644 · Cited by 24 · see citing works
METER-ML: A Multi-Sensor Earth Observation Benchmark for Automated Methane Source MappingPreprint
Zhu, B., Lui, N., Irvin, J., Le, J., Tadwalkar, S., Wang, C., Ouyang, Z., Liu, F. Y., Ng, A. Y., & Jackson, R. B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.11166 · Cited by 10 · see citing works
2020 IEEE GRSS Data Fusion Contest
Schmitt, M. (2024) · TIB Data Manager · doi:10.57702/pf0lp8uc · Cited by 10 · see citing works
Berg, P., Pham, M., & Courty, N. (2023) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss52108.2023.10283119 · Cited by 7 · see citing works
Mutual Guidance Meets Supervised Contrastive Learning: Vehicle Detection in Remote Sensing Images
Lê, H.-Â., Zhang, H., Pham, M., & Lefèvre, S. (2022) · Remote Sensing · doi:10.3390/rs14153689 · Cited by 5 · see citing works
Flagged first, then by citations
1 of the 21 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Kazanskiy, N. L., Khabibullin, R. M., Nikonorov, A. V., & Khonina, S. N. (2025) · Sensors · doi:10.3390/s25195965 · Cited by 78 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“By utilizing methods such as contrastive learning and self-supervised learning, the model can optimize the alignment between modalities, thereby enhancing the flow and interaction of information across them [182].”
Super-Resolution Cropland Mapping by Spectral and Spatial Training Samples Simulation
Jia, X., Hao, Z., Sun, L., Yang, Q., Wang, Z., Yin, Z., Shi, L., Li, X., Du, Y., & Ling, F. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3547042 · Cited by 2 · see citing works
He, Y., Zhang, X., 肖鹏峰, Li, Z., Muhtar, D., Gu, F., Liu, B., & Feng, P. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104876 · Cited by 1 · see citing works
Jeon, W., Yi, J., & Kim, Y. (2025) · Remote Sensing Letters · doi:10.1080/2150704x.2025.2610448 · Cited by 1 · see citing works
Insight into the Collocation of Multi-Source Satellite Imagery for Multi-Scale Vessel Detection
La, T. V., Pham, M., & Chini, M. (2024) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss53475.2024.10641796
Huang, Y., & Zhang, P. (2025) · International Conference on Geology, Mapping and Remote Sensing (ICGMRS) · doi:10.1109/icgmrs66001.2025.11065603
Machine Learning-Based Land Encroachment Detection Using Satellite Imagery
Malathi, S., Aarthi, D., Supreeth, B. R., Bhargavi, K., Muthumeena, S., & Tiwari, M. (2025) · International Conference on Recent Innovation in Science Engineering and Technology (ICRISET) · doi:10.1109/icriset64803.2025.11252304
Semi-Supervised Contrastive Representation Learning for Sunflower Phenology Estimation
Fatima, S. Z., Raees, M., Athar, U., Asif, M. B., Zafar, Z., Khurshid, H., Berns, K., & Fraz, M. M. (2025) · International Conference on Emerging Technologies · doi:10.1109/icet66147.2025.11321433
Augmenting X-ray astronomical representations with text
Vago, N. O. P., Tella, R. D., Cuesta-Lazaro, C., Smith, M. J., Garraffo, C., & Martínez-Galarza, J. R. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12343-1
Adversarial Robustness in Large Vision-Language Models
Anjali, K., Sahithi, K. V., & Jyotsna, S. (2026) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss69635.2026.11646060
- Authors
- Berg, P., Uzun, B., Pham, M.-T., & Courty, N.
- Affiliations
- Université de Bretagne Sud (FR)
- Venue
- IEEE Geoscience and Remote Sensing Letters
- Published
- 2024
- Volume
- 21
- Pages
- 1-5
- ISSN
- 1545-598X (ISSN-L)
- 1558-0571
- OpenAlex
- W4394674299
- Cited by
- 11
- References
- 21
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 1
- FWCI
- 1.82
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Geoscience and Remote Sensing Letters (publishedVersion)
- HAL (submittedVersion)
- Funders
- ANR AI Chair OTTOPIA (ANR-20-CHIA-0030)
- HPC Resources from GENCI-IDRIS (2023-AD011013514R1)
- Topics
- Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Remote Sensing and Land Use
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- supervised contrastive learning, satellite image classification, representation alignment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
