Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; 12 of the 28 sources its claims cite were checked for support.

SatMAE: Pre-Training Transformers for Temporal and Multi-Spectral Satellite Imagery

Open access
Cong, Yezhen, Khanna, Samar, Meng, Chenlin, Liu, Patrick, Rozi, Erik, He, Yutong, Burke, Marshall, Lobell, David, Ermon, Stefano · 2022 · Advances in Neural Information Processing Systems 35

Abstract

Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks. Developing similar techniques for satellite imagery presents significant opportunities as unlabelled data is plentiful and the inherent temporal and multi-spectral structure provides avenues to further improve existing pre-training strategies. In this paper, we present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). To leverage temporal information, we include a temporal embedding along with independently masking image patches across time. In addition, we demonstrate that encoding multi-spectral data as groups of bands with distinct spectral positional encodings is beneficial. Our approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to $\uparrow$ 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to $\uparrow$ 14%) and semantic segmentation. Code and data are available on the project website: https://sustainlab-group.github.io/SatMAE/

Overview

CoChat Summary

TL;DR

SatMAE adapts masked autoencoders to satellite imagery by adding temporal and spectral positional encodings and masking patches independently across time or spectral groups. It reports gains over prior self-supervised and supervised baselines on fMoW RGB, fMoW Sentinel, and several remote-sensing transfer tasks.

Key findings

  • On non-temporal fMoW RGB classification, SatMAE (ViT-Large) reaches 77.84 top-1 finetuned accuracy, a 6.29% improvement over the previous state-of-the-art self-supervised method GASSL (71.55).
  • On temporal fMoW RGB, SatMAE reaches 81.49 top-1 / 93.26 top-5, surpassing the previous state of the art by 4.48% and improving the non-temporal result by 2.06%; it also outperforms UTAE by 18%.
  • On fMoW Sentinel multi-spectral classification, SatMAE+Group+IM reaches 61.48 top-1 / 85.17 top-5, outperforming supervised training from scratch by ↑6.27% and ImageNet-initialized backbones by ↑4.84%.
  • Transfer learning improves land cover classification up to ↑14% (NAIP: SatMAE 71.77 vs. GASSL 57.63; EuroSAT: SatMAE+Group+IM 98.98 vs. SeCo 93.14), while BigEarthNet mAP is 82.13 and SpaceNet v1 building segmentation mIoU is 78.07.

Methods

  • SatMAE is based on Masked Autoencoder (MAE) with a ViT-Large backbone. It pre-trains on fMoW RGB temporal sequences and on a new fMoW Sentinel dataset of 712,874 training, 84,939 validation, and 84,966 test images using 13 Sentinel-2 bands.
  • For temporal data, it uses a temporal encoding from year, month, and hour, independently masks patches across time, and optionally uses inconsistent cropping and test-time augmentation. For multi-spectral data, it groups bands with separate patch embeddings and a spectral group encoding; experiments drop bands B1, B9, B10 and group the remaining 10 bands into RGB+NIR, Red Edge, and SWIR.
  • Evaluation covers fMoW RGB non-temporal and temporal classification, fMoW Sentinel classification, and transfer to NAIP, EuroSAT, BigEarthNet, and SpaceNet v1 segmentation, comparing against supervised scratch, ImageNet initialization, GASSL, SeCo, MoCo-v3, and UTAE.

Limitations

  • The paper states future work is needed on more efficient transformer architectures because longer token sequences strain computational resources, on optimal positional encodings and spectral band groupings, and on better ViT architectures for object detection and semantic segmentation.
  • The paper notes that GASSL and SeCo were pre-trained on a larger dataset (1M Sentinel-2 images vs. 713k) and with all 13 bands, so SatMAE may improve further with more data and longer pre-training.
  • The paper states that SatMAE could inherit biases from geographically imbalanced training data, especially toward Northern America and Europe, and advises caution when applying it to datasets with different geographic distributions. Evident from the reported tables, SatMAE does not surpass all baselines on every downstream task: BigEarthNet mAP is 82.13 vs. SeCo’s 82.62, and SpaceNet v1 mIoU is 78.07 vs. GASSL’s 78.51.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/068431-0015

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 1 Not valid · 8 Not checked · 69 Own results and interpretations

45 more claims · 1 with their cited sources checked

Create a free account to see all 88 claims and to verify the 4 references still open.

See all claims — free account
  • It may state two separate claims as one. 21
  • Probably not a substantial claim. 20
  • The quote only partly states this claim. 3

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 14 references

Research works printed but not linked: 14

  1. [1][1] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16000-16009, 2022. -
  2. [3][3] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9729-9738, 2020. -
  3. [15][15] Soroush Abbasi Koohpayegani, Ajinkya Tejankar, and Hamed Pirsiavash. Compress: Selfsupervised learning by compressing representations. Advances in Neural Information Processing Systems , 33:12980-12992, 2020. -
  4. [16][16] Abhishek Sinha, Jiaming Song, Chenlin Meng, and Stefano Ermon. D2c: Diffusion-decoding models for few-shot conditional generation. Advances in Neural Information Processing Systems , 34:12533-12548, 2021. -
  5. [18][18] Gencer Sumbul, Marcela Charfuelan, Begüm Demir, and Volker Markl. Bigearthnet: A largescale benchmark archive for remote sensing image understanding. In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium , pages 5901-5904. IEEE, 2019. -
  6. [29][29] Rose M Rustowicz, Robin Cheong, Lijing Wang, Stefano Ermon, Marshall Burke, and David Lobell. Semantic segmentation of crop type in africa: A novel dataset and analysis of deep learning methods. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages 75-82, 2019. -
  7. [31][31] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248-255, 2009. -
  8. [32][32] Yutong He, Dingjie Wang, Nicholas Lai, William Zhang, Chenlin Meng, Marshall Burke, David Lobell, and Stefano Ermon. Spatial-temporal super-resolution of satellite imagery via conditional pixel synthesis. Advances in Neural Information Processing Systems , 34:2790327915, 2021. -
  9. [46][46] Andrei Stoian, Vincent Poulain, Jordi Inglada, Victor Poughon, and Dawa Derksen. Land cover maps production with high resolution satellite image time series and convolutional neural networks: Adaptations and limits for operational systems. Remote Sensing , 11(17):1986, 2019. -
  10. [52][52] Wenyuan Li, Hao Chen, and Zhenwei Shi. Semantic segmentation of remote sensing images with self-supervised multitask representation learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 14:6438-6450, 2021. -
  11. [53][53] Wenyuan Li, Keyan Chen, Hao Chen, and Zhenwei Shi. Geographical knowledge-driven representation learning for remote sensing images. IEEE Transactions on Geoscience and Remote Sensing , 2021. -
  12. [57][57] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017. -
  13. [58][58] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning , pages 10347-10357. PMLR, 2021. -
  14. [61][61] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016. -

Cited by

See all 447

Metadata

Authors
Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D., & Ermon, S.
Venue
Advances in Neural Information Processing Systems 35
Published
2022
Pages
197-211
ISSN
1049-5258 (ISSN-L)
OpenAlex
W7133201458
Cited by
303
References
14
Citations by year
2026: 117 · 2025: 74 · 2024: 92 · 2023: 19 · 2022: 1
FWCI
62.17
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
  • Physical Sciences › Environmental Science › Ecology › Remote Sensing in Agriculture
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
multispectral satellite imagery, transformer pre-training, self-supervised learning, satellite image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: SatMAE: Pre-Training Transformers for Temporal and Multi-Spectral Satellite Imagery. https://app.cochat.ai/papers/10.52202/068431-0015
The paper
Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D., & Ermon, S. (2022). SatMAE: Pre-Training Transformers for Temporal and Multi-Spectral Satellite Imagery. Advances in Neural Information Processing Systems 35. https://doi.org/10.52202/068431-0015

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.