Very high resolution canopy height maps from RGB imagery using self-supervised vision transformer and convolutional decoder trained on aerial lidar
Open accessAbstract
Vegetation structure mapping is critical for understanding the global carbon cycle and monitoring nature-based approaches to climate adaptation and mitigation. Repeated measurements of these data allow for the observation of deforestation or degradation of existing forests, natural forest regeneration, and the implementation of sustainable agricultural practices like agroforestry. Assessments of tree canopy height and crown projected area at a high spatial resolution are also important for monitoring carbon fluxes and assessing tree-based land uses, since forest structures can be highly spatially heterogeneous, especially in agroforestry systems. Very high resolution satellite imagery (less than one meter (1 m) Ground Sample Distance) makes it possible to extract information at the tree level while allowing monitoring at a very large scale. This paper presents the first high-resolution canopy height map concurrently produced for multiple sub-national jurisdictions. Specifically, we produce very high resolution canopy height maps for the states of California and São Paulo, a significant improvement in resolution over the ten meter (10 m) resolution of previous Sentinel / GEDI based worldwide maps of canopy height. The maps are generated by the extraction of features from a self-supervised model trained on Maxar imagery from 2017 to 2020, and the training of a dense prediction decoder against aerial lidar maps. We also introduce a post-processing step using a convolutional network trained on GEDI observations. We evaluate the proposed maps with set-aside validation lidar data as well as by comparing with other remotely sensed maps and field-collected data, and find our model produces an average Mean Absolute Error (MAE) of 2.8 m and Mean Error (ME) of 0.6 m.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
High-Resolution Global Maps of 21st-Century Forest Cover Change
Hansen, M. C., Potapov, P. V., Moore, R., Hancher, M., Turubanova, S. A., Tyukavina, A., Thau, D., Stehman, S. V., Goetz, S. J., Loveland, T. R., et al. (2013) · Science · doi:10.1126/science.1244693 · Cited by 11,704 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · see citing works
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · see citing works
Road Extraction by Deep Residual U-Net
Zhang, Z., Liu, Q., & Wang, Y. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2802944 · Cited by 3,179 · cited 1× · see citing works
“We utilized a ResUNet-18 architecture for our baseline (Zhang et al., 2017), which is an encoder-decoder architecture predicting a N × N canopy height map from a 3 × N × N RGB image, with N = 256.”
MethodGood practices for estimating area and assessing accuracy of land change
Olofsson, P., Foody, G. M., Herold, M., Stehman, S. V., Woodcock, C. E., & Wulder, M. A. (2014) · Remote Sensing of Environment · doi:10.1016/j.rse.2014.02.015 · Cited by 3,134 · see citing works
2011 International Conference on Computer Vision
(2011) · doi:10.1109/iccv17851.2011 · Cited by 2,895 · see citing works
Vision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · see citing works
Old-growth forests as global carbon sinks
Luyssaert, S., Schulze, E., Börner, A., Knohl, A., Hessenmöller, D., Law, B. E., Ciais, P., & Grace, J. (2008) · Nature · doi:10.1038/nature07276 · Cited by 1,879 · see citing works
Friedlingstein, P., Jones, M. W., O’Sullivan, M., Andrew, R. M., Hauck, J., Peters, G. P., Peters, W., Pongratz, J., Sitch, S. A., Quéré, C. L., et al. (2019) · Earth system science data · doi:10.5194/essd-11-1783-2019 · Cited by 1,692 · see citing works
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
Luo, W., Li, Y., Urtasun, R., & Zemel, R. (2017) · arXiv · doi:10.48550/arxiv.1701.04128 · Cited by 1,575 · see citing works
Mapping global forest canopy height through integration of GEDI and Landsat data
Potapov, P. V., Li, X., Hernández-Serna, A., Tyukavina, A., Hansen, M. C., Kommareddy, A. R., Pickens, A., Turubanova, S. A., Tang, H., Silva, C. E., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.112165 · Cited by 1,476 · see citing works
Markus, T., Neumann, T., Martino, A. J., Abdalati, W., Brunt, K. M., Csatho, B. M., Farrell, S. L., Fricker, H. A., Gardner, A., Harding, D., et al. (2017) · Remote Sensing of Environment · doi:10.1016/j.rse.2016.12.029 · Cited by 1,421 · see citing works
Dubayah, R., Blair, J. B., Goetz, S. J., Fatoyinbo, T., Hansen, M. C., Healey, S. P., Hofton, M. A., HURTT, G. C., Kellner, J. R., Luthcke, S. B., et al. (2020) · Science of Remote Sensing · doi:10.1016/j.srs.2020.100002 · Cited by 1,292 · see citing works
Global maps of twenty-first century forest carbon fluxes
Harris, N. L., Gibbs, D. A., Baccini, A., Birdsey, R. A., de Bruin, S., Farina, M. K., Fatoyinbo, T., Hansen, M. C., Herold, M., Houghton, R. A., et al. (2021) · Nature Climate Change · doi:10.1038/s41558-020-00976-6 · Cited by 1,260 · see citing works
DINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078 · cited 3× · see citing works
“Following the recent success of self-supervised learning on dense prediction tasks from Oquab et al. (2023), we employed a self-supervised learning step on 18 million globally distributed, randomly sampled 256 × 256 pixel Maxar satellite images to obtain an image encoder delivering features…”
BackgroundMethodSelf-citationRate of tree carbon accumulation increases continuously with tree size
Stephenson, N. L., Das, A. J., Condit, R., Russo, S. E., Baker, P. J., Beckman, N. G., Coomes, D. A., Lines, E. R., Morris, W. K., Rüger, N., et al. (2014) · Nature · doi:10.1038/nature12914 · Cited by 1,051 · see citing works
AdaBins: Depth Estimation Using Adaptive Bins
Bhat, S. F., Alhashim, I., & Wonka, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00400 · Cited by 862 · see citing works
A high-resolution canopy height model of the Earth
Lang, N., Jetz, W., Schindler, K., & Wegner, J. D. (2023) · Nature Ecology & Evolution · doi:10.1038/s41559-023-02206-6 · Cited by 810 · see citing works
Mapping carbon accumulation potential from global natural forest regrowth
Cook‐Patton, S. C., Leavitt, S. M., Gibbs, D. A., Harris, N. L., Lister, K., Anderson‐Teixeira, K. J., Briggs, R. D., Chazdon, R. L., Crowther, T. W., Ellis, P. W., et al. (2020) · Nature · doi:10.1038/s41586-020-2686-x · Cited by 638 · see citing works
An unexpectedly large count of trees in the West African Sahara and Sahel
Brandt, M., Tucker, C. J., Kariryaa, A., Rasmussen, K., Abel, C., Small, J., Chave, J., Rasmussen, L. V., Hiernaux, P., Diouf, A. A., et al. (2020) · Nature · doi:10.1038/s41586-020-2824-5 · Cited by 466 · see citing works
The Brazilian Atlantic Forest: A Shrinking Biodiversity Hotspot
Ribeiro, M. C., Martensen, A. C., Metzger, J. P., Tabarelli, M., Scarano, F. R., & Fortin, M. (2011) · Springer eBooks · doi:10.1007/978-3-642-20992-5_21 · Cited by 406 · see citing works
Generating Pit-free Canopy Height Models from Airborne Lidar
Khosravipour, A., Skidmore, A. K., Isenburg, M., Wang, T., & Hussin, Y. A. (2014) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.80.9.863 · Cited by 372 · see citing works
Humid tropical forest disturbance alerts using Landsat data
Hansen, M. C., Krylov, A., Tyukavina, A., Potapov, P. V., Turubanova, S., Zutta, B., Ifo, S., Margono, B., Stolle, F., & Moore, R. (2016) · Environmental Research Letters · doi:10.1088/1748-9326/11/3/034008 · Cited by 303 · see citing works
Hancock, S., Armston, J. D., Hofton, M. A., Sun, X., Tang, H., Duncanson, L. I., Kellner, J. R., & Dubayah, R. (2019) · Earth and Space Science · doi:10.1029/2018ea000506 · Cited by 303 · see citing works
Lang, N., Kalischek, N., Armston, J. D., Schindler, K., Dubayah, R., & Wegner, J. D. (2021) · Remote Sensing of Environment · doi:10.1016/j.rse.2021.112760 · Cited by 303 · see citing works
Li, W., Niu, Z., Shang, R., Qin, Y., Wang, L., & Chen, H. Y. H. (2020) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2020.102163 · Cited by 280 · see citing works
Duncanson, L. I., Neuenschwander, A., Hancock, S., Thomas, N. M., Fatoyinbo, T., Simard, M., Silva, C. A., Armston, J. D., Luthcke, S. B., Hofton, M. A., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.111779 · Cited by 278 · see citing works
Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning
Reed, C., Gupta, R., Li, S., Brockman, S., Funk, C., Clipp, B., Keutzer, K., Candido, S., Uyttendaele, M., & Darrell, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00378 · Cited by 276 · see citing works
Stehman, S. V. (2014) · International Journal of Remote Sensing · doi:10.1080/01431161.2014.930207 · Cited by 257 · see citing works
Fusing simulated GEDI, ICESat-2 and NISAR data for regional aboveground biomass mapping
Silva, C. A., Duncanson, L. I., Hancock, S., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.112234 · Cited by 232 · see citing works
Efficient Transformer for Remote Sensing Image Segmentation
Xu, Z., Zhang, W., Zhang, T., Yang, Z., & Li, J. (2021) · Remote Sensing · doi:10.3390/rs13183585 · Cited by 229 · see citing works
Camarretta, N., Harrison, P. A., Bailey, T. G., Potts, B. M., Lucieer, A., Davidson, N. J., & Hunt, M. (2019) · New Forests · doi:10.1007/s11056-019-09754-5 · Cited by 225 · see citing works
Continental-Scale Building Detection from High Resolution Satellite ImageryPreprint
Sirko, W., Kashubin, S., Ritter, M., Annkah, A., Bouchareb, Y. S. E., Dauphin, Y., Keysers, D., Neumann, M., Cissé, M., & Quinn, J. A. (2021) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2023.35 · Cited by 212 · see citing works
Miangoleh, S. M. H., Dille, S., Mai, L., Paris, S., & Aksoy, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00956 · Cited by 186 · see citing works
High aboveground carbon stock of African tropical montane forests
Cuní‐Sanchez, A., Sullivan, M. J. P., Platts, P. J., Lewis, S. L., Marchant, R., Imani, G., Hubau, W., Abiem, I., Adhikari, H., Albrecht, T., et al. (2021) · Nature · doi:10.1038/s41586-021-03728-4 · Cited by 166 · see citing works
The overlooked contribution of trees outside forests to tree cover and woody biomass across Europe
Liu, S., Brandt, M., Nord‐Larsen, T., Chave, J., Reiner, F., Lang, N., Tong, X., Ciais, P., Igel, C., Pascual, A., et al. (2023) · Science Advances · doi:10.1126/sciadv.adh4097 · Cited by 157 · see citing works
Adam, M., Urbazaev, M., Dubois, C., & Schmullius, C. (2020) · Remote Sensing · doi:10.3390/rs12233948 · Cited by 151 · see citing works
Nation-wide mapping of tree-level aboveground carbon stocks in Rwanda
Mugabowindekwe, M., Brandt, M., Chave, J., Reiner, F., Skole, D. L., Kariryaa, A., Igel, C., Hiernaux, P., Ciais, P., Mertz, O., et al. (2022) · Nature Climate Change · doi:10.1038/s41558-022-01544-w · Cited by 144 · see citing works
Monocular depth estimation with hierarchical fusion of dilated CNNs and soft-weighted-sum inference
Li, B., Dai, Y., & He, M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.05.029 · Cited by 142 · see citing works
Monitoring tropical forest carbon stocks and emissions using Planet satellite data
Csillik, O., Kumar, P., Mascaro, J., O’Shea, T., & Asner, G. P. (2019) · Scientific Reports · doi:10.1038/s41598-019-54386-6 · Cited by 140 · see citing works
Maxwell, A. E., Warner, T. A., & Guillén, L. A. (2021) · Remote Sensing · doi:10.3390/rs13132591 · Cited by 106 · see citing works
Revisiting Weakly Supervised Pre-Training of Visual Perception Models
Singh, M., Gustafson, L., Adcock, A., De Freitas Reis, V., Gedik, B., Kosaraju, R. P., Mahajan, D., Girshick, R., Dollar, P., & Van Der Maaten, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00088 · Cited by 99 · see citing works
Schwartz, M., Ciais, P., Ottlé, C., Truchis, A. D., Vega, C., Fayad, I., Brandt, M., Fensholt, R., Baghdadi, N. N., Morneau, F., et al. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103711 · Cited by 95 · see citing works
Sub-meter tree height mapping of California using aerial images and LiDAR-informed U-Net model
Wagner, F. H., Roberts, S., Ritz, A. L., Carter, G., Dalagnol, R., Favrichon, S., Hirye, M. C. M., Brandt, M., Ciais, P., & Saatchi, S. S. (2024) · Remote Sensing of Environment · doi:10.1016/j.rse.2024.114099 · Cited by 77 · see citing works
Astola, H., Seitsonen, L., Halme, E., Molinier, M., & Lönnqvist, A. (2021) · Remote Sensing · doi:10.3390/rs13122392 · Cited by 71 · see citing works
Improving uncertainty in forest carbon accounting for REDD+ mitigation efforts
Yanai, R. D., Wayson, C. A., Lee, D., Espejo, A. B., Campbell, J. L., Green, M. B., Zukswert, J. M., Yoffe, S. B., Aukema, J. E., Lister, A. J., et al. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/abb96f · Cited by 68 · see citing works
Weinstein, B. G., Graves, S., Marconi, S., SINGH, A. K., Zare, A., Stewart, D., Bohlman, S., & White, E. P. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009180 · Cited by 61 · see citing works
Monitoring and Evaluating Forest Restoration Success
Vallauri, D., Aronson, J. C., Dudley, N., & Vallejo, V. R. (2006) · Springer eBooks · doi:10.1007/0-387-29112-1_21 · Cited by 55 · see citing works
Gibril, M. B. A., Shafri, H. Z. M., Al-Ruzouq, R. I., et al. (2023) · Drones · doi:10.3390/drones7020093 · Cited by 55 · see citing works
Monitoring Young Tropical Forest Restoration Sites: How Much to Measure?
Viani, R. A. G., Barreto, T. E., Farah, F. T., Rodrigues, R. R., & Brancalion, P. H. S. (2018) · Tropical Conservation Science · doi:10.1177/1940082918780916 · Cited by 52 · see citing works
A ViT-Based Multiscale Feature Fusion Approach for Remote Sensing Image Segmentation
Wang, W., Tang, C., Wang, X., et al. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3187135 · Cited by 44 · see citing works
Tesfay, F., Moges, Y., & Asfaw, Z. (2022) · International Journal of Forestry Research · doi:10.1155/2022/4729336 · Cited by 33 · see citing works
Skole, D. L., Samek, J. H., Dieng, M., & Mbow, C. (2021) · Forests · doi:10.3390/f12121652 · Cited by 31 · see citing works
The hunt for the world’s missing carbon
Popkin, G. (2015) · Nature · doi:10.1038/523020a · Cited by 29 · see citing works
LiDAR Surveys over Selected Forest Research Sites, Brazilian Amazon, 2008-2018
dos-Santos, M. N., Keller, M., & Morton, D. C. (2019) · Oak Ridge National Laboratory Distributed Active Archive Center for Biogeochemical Dynamics · doi:10.3334/ornldaac/1644 · Cited by 18 · see citing works
Schacher, A., Roger, E., Williams, K. J., Stenson, M., Sparrow, B. D., & Lacey, J. (2023) · Remote Sensing · doi:10.3390/rs15051407 · Cited by 15 · see citing works
A Compromise Principle in Deep Monocular Depth EstimationPreprint
Fu, H., Gong, M., Wang, C., & Tao, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.08267 · Cited by 11 · see citing works
MapBiomas initiative: Mapping annual land cover and land use changes in Brazil from 1985 to 2017.
Azevedo, T., Souza, C., Shimbo, J. Z., & Alencar, A. A. C. (2018) · AGUFM · Cited by 10 · see citing works
Timber Exploitation in Colonial Brazil: A Historical Perspective of the Atlantic Forest
Maioli, V., Belharte, S., Kropf, M. S., & Callado, C. H. (2020) · Historia Ambiental Latinoamericana y Caribeña (HALAC) revista de la Solcha · doi:10.32991/2237-2717.2020v10i2.p74-101 · Cited by 8 · see citing works
da Luz, N. B., Garrastazú, M. C., Rosot, M. A. D., Maran, J. C., de Oliveira, Y. M. M., Franciscon, L., Cardoso, D. J., & de Freitas, J. V. (2018) · Pesquisa Florestal Brasileira · doi:10.4336/2018.pfb.38e201701493 · Cited by 7 · see citing works
Vision Transformers, a new approach for high-resolution and large-scale mapping of canopy heightsPreprint
Fayad, I., Ciais, P., Schwartz, M., et al. (2023) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.48550/arxiv.2304.11487 · Cited by 7 · see citing works
GEDI L3 Gridded Land Surface Metrics, Version 1
Dubayah, R., Luthcke, S. B., Sabaka, T. J., Nicholas, J. B., Preaux, S. A., & Hofton (2021) · Oak Ridge National Laboratory Distributed Active Archive Center for Biogeochemical Dynamics · doi:10.3334/ornldaac/1865 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- da Luz, 2018, Inventário florestal nacional do brasil - uma abordagem em escala de paisagem para monitorar e avaliar paisagens florestais, Pesquisa Florestal Bras., 38
- Reytar, 2020, Measuring progress in forest and landscape restoration, Unasylva, 71, 62
Cited by
Sorted by citations
Moudrý, V., Gábor, L., Marselis, S., Pracná, P., Barták, V., Prošek, J., Navrátilová, B., Novotný, J., Potůčková, M., Gdulová, K., et al. (2024) · Ecosphere · doi:10.1002/ecs2.70026 · Cited by 43 · see citing works
- Authors
- Tolan, J., Yang, H.-I., Nosarzewski, B., Couairon, G., Vo, H. V., Brandt, J., Spore, J., Majumdar, S., Haziza, D., Vamaraju, J., Moutakanni, T., Bojanowski, P., Johns, T., White, B., Tiecke, T., & Couprie, C.
- Affiliations
- Meta (United States) (US)
- World Resources Institute (US)
- Desert Research Institute (US)
- Corresponding author
- Brandt, John
- Venue
- Remote Sensing of Environment
- Publisher
- Elsevier BV
- Published
- 2024-01
- Volume
- 300
- Pages
- 113888
- ISSN
- 0034-4257 (ISSN-L)
- OpenAlex
- W4388448546
- Cited by
- 419
- References
- 64
- Citations by year
- 2026: 216 · 2025: 158 · 2024: 44 · 2023: 1
- FWCI
- 73.23
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Remote Sensing of Environment (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Environmental Science › Environmental Engineering › Remote Sensing and LiDAR Applications
- Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
- Physical Sciences › Environmental Science › Ecology › Remote Sensing in Agriculture
- Keywords
- canopy height mapping, very high resolution satellite imagery, RGB images, self-supervised vision transformers, convolutional decoder, airborne laser scanning, GEDI, agroforestry, forest structure, California, São Paulo, forest carbon monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
