The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Vision Transformers for Remote Sensing Image Classification
Open accessAbstract
In this paper, we propose a remote-sensing scene-classification method based on vision transformers. These types of networks, which are now recognized as state-of-the-art models in natural language processing, do not rely on convolution layers as in standard convolutional neural networks (CNNs). Instead, they use multihead attention mechanisms as the main building block to derive long-range contextual relation between pixels in images. In a first step, the images under analysis are divided into patches, then converted to sequence by flattening and embedding. To keep information about the position, embedding position is added to these patches. Then, the resulting sequence is fed to several multihead attention layers for generating the final representation. At the classification stage, the first token sequence is fed to a softmax classification layer. To boost the classification performance, we explore several data augmentation strategies to generate additional data for training. Moreover, we show experimentally that we can compress the network by pruning half of the layers while keeping competing classification accuracies. Experimental results conducted on different remote-sensing image datasets demonstrate the promising capability of the model compared to state-of-the-art methods. Specifically, Vision Transformer obtains an average classification accuracy of 98.49%, 95.86%, 95.56% and 93.83% on Merced, AID, Optimal31 and NWPU datasets, respectively. While the compressed version obtained by removing half of the multihead attention layers yields 97.90%, 94.27%, 95.30% and 93.05%, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikainen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,584
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400
Bag-of-visual-words and spatial extensions for land-use classification
Yang, Y., & Newsam, S. (2010) · Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems · doi:10.1145/1869790.1869829 · Cited by 2,969
Remote Sensing Image Scene Classification: Benchmark and State of the Art
Cheng, G., Han, J., & Lu, X. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2675998 · Cited by 2,926
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00020 · Cited by 2,733
Deep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
AID: A benchmark data set for performance evaluation of aerial scene classification
Xia, G.-S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., & Lu, X. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2685945 · Cited by 2,372
Cheng, G., Yang, C., Yao, X., Guo, L., & Han, J. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2783902 · Cited by 1,226
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.521 · Cited by 1,212
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q., Vaswani, A., & Shlens, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,205
Convolutional neural networks for large-scale remote-sensing image classification
Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2612821 · Cited by 1,114
Discovering objects and their location in images
Sivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.77 · Cited by 975
Generative Adversarial Networks for Hyperspectral Image Classification
Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2805286 · Cited by 734
Remote sensing platforms and sensors: A survey
Toth, C., & Jóźków, G. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2015.10.004 · Cited by 685
Learning Deep Transformer Models for Machine Translation
Wang, Q., Li, B., Xiao, T., Zhu, J., Li, C., Wong, D. F., & Chao, L. S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1176 · Cited by 673
Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks
Marmanis, D., Datcu, M., Esch, T., & Stilla, U. (2016) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2499239 · Cited by 646
Scene Classification With Recurrent Attention of VHR Remote Sensing Images
Wang, Q., Liu, S., Chanussot, J., & Li, X. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2864987 · Cited by 613
Advances in Unmanned Aerial Vehicles
Valavanis, K. P. (2007) · Springer eBooks · doi:10.1007/978-1-4020-6114-1 · Cited by 547
He, J., Zhao, L., Yang, H., Zhang, M., & Li, W. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2934760 · Cited by 383
Exploring the Use of Google Earth Imagery and Object-Based Methods in Land Use/Cover Mapping
Hu, Q., Wu, W., Xia, T., Yu, Q., Yang, P., Li, Z., & Song, Q. (2013) · Remote Sensing · doi:10.3390/rs5116026 · Cited by 309
Nonlocal Graph Convolutional Networks for Hyperspectral Image Classification
Mou, L., Lu, X., Li, X., & Zhu, X. X. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2973363 · Cited by 291
Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling
He, N., Fang, L., Li, S., Plaza, A., & Plaza, J. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2845668 · Cited by 284
Remote Sensing Scene Classification by Gated Bidirectional Network
Sun, H., Li, S., Zheng, X., & Lu, X. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2931801 · Cited by 224
Hu, W.-S., Li, H.-C., Pan, L., Li, W., Tao, R., & Du, Q. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2961947 · Cited by 218
Land Cover Classification from fused DSM and UAV Images Using Convolutional Neural Networks
Al-Najjar, H. A. H., Kalantar, B., Pradhan, B., Saeidi, V., Halin, A. A., Ueda, N., & Mansor, S. (2019) · Remote Sensing · doi:10.3390/rs11121461 · Cited by 206
Efficient Saliency-Based Object Detection in Remote Sensing Images Using Deep Belief Networks
Diao, W., Sun, X., Zheng, X., Dou, F., Wang, H., & Fu, K. (2016) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2498644 · Cited by 182
A Multiple-Instance Densely-Connected ConvNet for Aerial Scene Classification
Bi, Q., Qin, K., Li, Z., Zhang, H., Xu, K., & Xia, G.-S. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975718 · Cited by 166
Style Augmentation: Data Augmentation via Style Randomization
Jackson, P. T., Atapour–Abarghouei, A., Bonner, S., Breckon, T. P., & Obara, B. (2018) · Durham Research Online (Durham University) · doi:10.48550/arxiv.1809.05375 · Cited by 158
Huang, L., Chen, C., Li, W., & Du, Q. (2016) · Remote Sensing · doi:10.3390/rs8060483 · Cited by 155
Feng, J., Yu, H., Wang, L., Cao, X., Zhang, X., & Jiao, L. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2899057 · Cited by 153
Non-Autoregressive Transformer for Speech Recognition
Chen, N., Watanabe, S., Villalba, J., Żelasko, P., & Dehak, N. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3044547 · Cited by 137
A Two-Stream Deep Fusion Framework for High-Resolution Aerial Scene Classification
Yu, Y., & Liu, F. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/8639367 · Cited by 136
Attention GANs: Unsupervised Deep Feature Learning for Aerial Scene Classification
Yu, Y., Li, X., & Liu, F. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2937830 · Cited by 123
Scene Classification Based on Two-Stage Deep Feature Fusion
Liu, Y., Liu, Y., & Ding, L. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2779469 · Cited by 114
Microscopic Traffic Data Collection by Remote Sensing
Hoogendoorn, S. P., van Zuylen, H. J., Schreuder, M., Gorte, B. G. H., & George Vosselman, A. (2003) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/1855-15 · Cited by 107
Bazi, Y., Rahhal, M. M. A., Alhichri, H. S., et al. (2019) · Remote Sensing · doi:10.3390/rs11242908 · Cited by 91
Self-citationAggregated Deep Local Features for Remote Sensing Image Retrieval
Imbriaco, R., Sebastian, C., Bondarev, E., & de With, P. H. N. (2019) · Remote Sensing · doi:10.3390/rs11050493 · Cited by 83
Aggregated deep local features for remote sensing image retrieval
Imbriaco, R., Sebastian, C. C., Bondarev, Y. E., & With, P. P. D. (2019) · TU/e Research Portal · Cited by 71
Daldegan, G. A., Roberts, D. A., & Ribeiro, F. D. F. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.111340 · Cited by 70
Remote Sensing Scene Classification Based on Multi-Structure Deep Features Fusion
Xue, W., Dai, X., & Liu, L. (2020) · IEEE Access · doi:10.1109/access.2020.2968771 · Cited by 57
Improving Spatial Feature Representation from Aerial Scenes by Using Convolutional Networks
Nogueira, K., Miranda, W. O., & Santos, J. A. D. (2015) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi.2015.39 · Cited by 51
Recurrent neural networks for remote sensing image classification
Lakhal, M. I., Çevıkalp, H., Escalera, S., & Ofli, F. (2018) · IET Computer Vision · doi:10.1049/iet-cvi.2017.0420 · Cited by 40
Li, Q., Qi, S., Shen, Y., et al. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2412955 · Cited by 29
Real-time scene understanding for UAV imagery based on deep convolutional neural networks
Sheppard, C., & Rahnemoonfar, M. (2017) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2017.8127435 · Cited by 29
Liu, T., Abd-Elrahman, A. H., Zare, A., Dewitt, B. A., Flory, S. L., & Smith, S. E. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.06.031 · Cited by 28
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019) · arXiv · doi:10.48550/arxiv.1901.02860
Cited by
Wang, L., Li, R., Zhang, C., Fang, S., Duan, C., Meng, X., & Atkinson, P. M. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.06.008 · Cited by 1,349
Large Selective Kernel Network for Remote Sensing Object Detection
Li, Y., Hou, Q., Zheng, Z., Cheng, M.-M., Yang, J., & Li, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01540 · Cited by 893
Upadhyay, A., Chandel, N. S., Singh, K. P., Chakraborty, S. K., Nandede, B. M., Kumar, M., Subeesh, A., Upendar, K., Salem, A., & Elbeltagi, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11100-x · Cited by 468
Transformers in Remote Sensing: A Survey
Aleissaee, A. A., Kumar, A., Anwer, R. M., Khan, S., Cholakkal, H., Xia, G.-S., & Khan, F. S. (2023) · Remote Sensing · doi:10.3390/rs15071860 · Cited by 417
Liu, M., Chai, Z., Deng, H., & Liu, R. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3177235 · Cited by 409
Metadata
- Authors
- Bazi, Y., Bashmal, L., Rahhal, M. M. A., Dayil, R. A., & Ajlan, N. A.
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2021-02-01
- Volume
- 13
- Issue
- 3
- Pages
- 516
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W3128592650
- Cited by
- 637
- References
- 49
- FWCI
- 41.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.