The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery
Abstract
This article presents a transformer and convolutional neural network (CNN) hybrid deep neural network for semantic segmentation of very high resolution (VHR) remote sensing imagery. The model follows an encoder–decoder structure. The encoder module uses a new universal backbone Swin transformer to extract features to achieve better long-range spatial dependencies modeling. The decoder module draws on some effective blocks and successful strategies of CNN-based models in remote sensing image segmentation. In the middle of the framework, an atrous spatial pyramid pooling block based on depthwise separable convolution (SASPP) is applied to obtain a multiscale context. A U-shaped decoder is used to gradually restore the size of the feature maps. Three skip connections are built between the encoder and decoder feature maps of the same size to maintain the transmission of local details and enhance the communication of multiscale features. A squeeze-and-excitation (SE) channel attention block is added before segmentation for feature augmentation. An auxiliary boundary detection branch is combined to provide edge constraints for semantic segmentation. Extensive ablation experiments were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam benchmarks to test the effectiveness of multiple components of the network. At the same time, the proposed method is compared with the current state-of-the-art methods on the two benchmarks. The proposed hybrid network achieved the second highest overall accuracy (OA) on both the Potsdam and Vaihingen benchmarks (code and models are available athttps://github.com/zq7734509/mmsegmentation-multilayer).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Chen, Q., Wang, L., Wu, Y., Wu, G., Guo, Z., & Waslander, S. L. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.11.011 · Cited by 150
RetractedDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Dual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351
Deep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 4×
“SETR [18] applies the transformer structure to the semantic segmentation task; it uses the transformer structure departed from the ViT as encoder and obtains dense predicted results by applying three different CNN-based decoders.”
InfluentialRethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.15840 · Cited by 4,004
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes
Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.-W., & Heng, P.-A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2845918 · Cited by 2,473
ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data
Diakogiannis, F. I., Waldner, F., Caccetta, P., & Wu, C. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.01.013 · Cited by 2,153
ResNeSt: Split-Attention Networks
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.08955 · Cited by 1,809 · cited 2×
“ResNeSt [22] combines the squeeze-and-excitation (SE) attention block on the basis of the ResNeXt’s cardinality to enhance the interaction of channel information.”
Weighted Res-UNet for High-Quality Retina Vessel Segmentation
Xiao, X., Lian, S., Luo, Z., & Li, S. (2018) · 2018 9th International Conference on Information Technology in Medicine and Education (ITME) · doi:10.1109/itme.2018.00080 · Cited by 963
Learning a Discriminative Feature Network for Semantic Segmentation
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00199 · Cited by 870
Classification with an edge: Improving semantic image segmentation with boundary detection
Marmanis, D., Schindler, K., Wegner, J. D., Galliani, S., Datcu, M., & Stilla, U. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.11.009 · Cited by 670
Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks
Volpi, M., & Tuia, D. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2616585 · Cited by 436
UPSNet: A Unified Panoptic Segmentation Network
Xiong, Y., Liao, R., Zhao, H., Hu, R., Bai, M., Yumer, E., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00902 · Cited by 402
Hoeser, T., & Kuenzer, C. (2020) · Remote Sensing · doi:10.3390/rs12101667 · Cited by 366
Disentangled Non-local Neural Networks
Yin, M., Yao, Z., Cao, Y., Li, X., Zhang, Z., Lin, S., & Hu, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_12 · Cited by 321
Semantic labeling in very high resolution images via a self-cascaded convolutional neural network
Liu, Y., Fan, B., Wang, L., Bai, J., Xiang, S., & Pan, C. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.12.007 · Cited by 290
Efficient structure from motion for large-scale UAV images: A review and a comparison of SfM tools
Jiang, S., Jiang, C., & Jiang, W. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.04.016 · Cited by 286
Self-citationLearning Joint Spatial-Temporal Transformations for Video Inpainting
Zeng, Y., Fu, J., & Chao, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58517-4_31 · Cited by 281
SEMANTIC SEGMENTATION OF AERIAL IMAGES WITH AN ENSEMBLE OF CNNS
Marmanis, D., Wegner, J. D., Galliani, S., Schindler, K., Datcu, M., & Stilla, U. (2016) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprs-annals-iii-3-473-2016 · Cited by 268
Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution Images
Wang, H., Wang, Y., Zhang, Q., Xiang, S., & Pan, C. (2017) · Remote Sensing · doi:10.3390/rs9050446 · Cited by 201
TreeUNet: Adaptive Tree convolutional neural networks for subdecimeter aerial image segmentation
Yue, K., Yang, L., Li, R., Hu, W., Zhang, F., & Li, W. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.07.007 · Cited by 189
An attention-fused network for semantic segmentation of very-high-resolution remote sensing imagery
Yang, X., Li, S., Chen, Z., Chanussot, J., Jia, X., Zhang, B., Li, B., & Chen, P. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.05.004 · Cited by 189
AlignSeg: Feature-Aligned Segmentation Networks
Huang, Z., Wei, Y., Wang, X., Shi, H., Liu, W., & Huang, T. S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3062772 · Cited by 182
Mou, L., & Zhu, X. X. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2841808 · Cited by 174
Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks
Nogueira, K., Mura, M. D., Chanussot, J., Schwartz, W. R., & dos Santos, J. A. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2913861 · Cited by 164
Ding, L., Zhang, J., & Bruzzone, L. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2964675 · Cited by 157
Boundary Loss for Remote Sensing Imagery Semantic Segmentation
Bokhovkin, A., & Burnaev, E. V. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22808-8_38 · Cited by 152
Unmixing Convolutional Features for Crisp Edge Detection
Huan, L., Xue, N., Zheng, X., He, W. P., Gong, J., & Xia, G.-S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3084197 · Cited by 133
ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images
Liu, S., Ding, W., Liu, C., Liu, Y., Wang, Y., & Li, H. (2018) · Remote Sensing · doi:10.3390/rs10091339 · Cited by 100
Supervised Classification of Multisensor Remotely Sensed Images Using a Deep Learning Framework
Piramanayagam, S., Saber, E. S., Schwartzkopf, W. C., & Koehler, F. W. (2018) · Remote Sensing · doi:10.3390/rs10091429 · Cited by 95
Zhang, C., Jiang, W.-S., & Zhao, Q. (2021) · Remote Sensing · doi:10.3390/rs13061176 · Cited by 21
Self-citation
Cited by
Wang, L., Li, R., Zhang, C., Fang, S., Duan, C., Meng, X., & Atkinson, P. M. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.06.008 · Cited by 1,349
Self-citationTransformers in Remote Sensing: A Survey
Aleissaee, A. A., Kumar, A., Anwer, R. M., Khan, S., Cholakkal, H., Xia, G.-S., & Khan, F. S. (2023) · Remote Sensing · doi:10.3390/rs15071860 · Cited by 417
Wu, H., Huang, P., Zhang, M., Tang, W., & Yu, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3314641 · Cited by 360
RS-Mamba for Large Remote Sensing Image Dense Prediction
Zhao, S., Chen, H., Zhang, X., Xiao, P., Bai, L., & Ouyang, W. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3425540 · Cited by 323
Metadata
- Authors
- Zhang, C., Jiang, W., Zhang, Y., Wang, W., Zhao, Q., & Wang, C.
- Venue
- IEEE Transactions on Geoscience and Remote Sensing
- Published
- 2022
- Volume
- 60
- Pages
- 1-20
- ISSN
- 0196-2892 (ISSN-L)
- 1558-0644
- OpenAlex
- W4205365435
- Cited by
- 325
- References
- 49
- FWCI
- 28.65
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.