Towards better exploiting convolutional neural networks for remote sensing scene classification
Open accessThe DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
We present an analysis of three possible strategies for exploiting the power of existing convolutional neural networks (ConvNets or CNNs) in different scenarios from the ones they were trained: full training, fine tuning, and using ConvNets as feature extractors. In many applications, especially including remote sensing, it is not feasible to fully design and train a new ConvNet, as this usually requires a considerable amount of labeled data and demands high computational costs. Therefore, it is important to understand how to better use existing ConvNets. We perform experiments with six popular ConvNets using three remote sensing datasets. We also compare ConvNets in each strategy with existing descriptors and with state-of-the-art baselines. Results point that fine tuning tends to be the best performing strategy. In fact, using the features from the fine-tuned ConvNet with linear SVM obtains the best results. We also achieved state-of-the-art results for the three datasets used.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 9×
“GoogLeNet, presented in [30], is the ConvNet architecture that won the ILSVRC-2014 competition (classification and detection tracks) while VGG ConvNets, presented in [29], won the localization and classification tracks of the same competition.”
BackgroundMethodInfluentialDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Cited by
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762307 · Cited by 3,400
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification
Helber, P., Bischke, B., Dengel, A., & Borth, D. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2918242 · Cited by 1,951
“As shown in previous work [6], [15], [17], [19], deep CNNs have been shown to outperform non-deep learning approaches in land use and land cover image classification.”
BackgroundMethodResultInfluentialGoogle Earth Engine for geo-big data applications: A meta-analysis and systematic review
Tamiminia, H., Salehi, B., Mahdianpari, M., Quackenbush, L., Adeli, S., & Brisco, B. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.04.001 · Cited by 1,363
Cheng, G., Yang, C., Yao, X., Guo, L., & Han, J. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2783902 · Cited by 1,226
Shen, C. (2018) · Water Resources Research · doi:10.1029/2018wr022643 · Cited by 1,133
Deep learning classifiers for hyperspectral imaging: A review
Paoletti, M. E., Haut, J. M., Plaza, J., & Plaza, A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.09.006 · Cited by 1,073
Cheng, G., Xie, X., Han, J., Guo, L., & Xia, G.-S. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3005403 · Cited by 995
Ball, J. E., Anderson, D. T., & Chan, C. S. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.042609 · Cited by 746
Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks
Audebert, N., Le Saux, B., & Lefèvre, S. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.11.011 · Cited by 655
Scene Classification With Recurrent Attention of VHR Remote Sensing Images
Wang, Q., Liu, S., Chanussot, J., & Li, X. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2864987 · Cited by 613
Binary Neural Networks: A Survey
Qin, H., Gong, R., Liu, X., Bai, X., Song, J., & Sebe, N. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107281 · Cited by 546
Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images
Saha, S., Bovolo, F., & Bruzzone, L. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2886643 · Cited by 523
Deep learning for remote sensing image classification: A survey
Li, Y., Zhang, H., Xue, X., Jiang, Y., & Shen, Q. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1264 · Cited by 472
Deep Feature Fusion for VHR Remote Sensing Scene Classification
Chaib, S., Liu, H., Gu, Y., & Yao, H. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2700322 · Cited by 461
Vali, A., Comai, S., & Matteucci, M. (2020) · Remote Sensing · doi:10.3390/rs12152495 · Cited by 455
Joint deep learning for land cover and land use classification
Zhang, C., Sargent, I., Pan, X., Li, H., Gardiner, A., Hare, J., & Atkinson, P. M. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.11.014 · Cited by 451
Mahdianpari, M., Salehi, B., Rezaee, M., Mohammadimanesh, F., & Zhang, Y. (2018) · Remote Sensing · doi:10.3390/rs10071119 · Cited by 440
Cheng, G., Wang, Y., Xu, S., Wang, H., Xiang, S., & Pan, C. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2669341 · Cited by 437
An object-based convolutional neural network (OCNN) for urban land use classification
Zhang, C., Sargent, I., Pan, X., Li, H., Gardiner, A., Hare, J., & Atkinson, P. M. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.06.034 · Cited by 436
Remote Sensing Image Scene Classification Using CNN-CapsNet
Zhang, W., Tang, P., & Zhao, L. (2019) · Remote Sensing · doi:10.3390/rs11050494 · Cited by 425
Deep convolution neural network for image recognition
Traore, B. B., Kamsu-Foguem, B., & Tangara, F. (2018) · Ecological Informatics · doi:10.1016/j.ecoinf.2018.10.002 · Cited by 417
Alshehhi, R., Marpu, P. R., Woon, W. L., & Mura, M. D. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.05.002 · Cited by 407
Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks
Audebert, N., Le Saux, B., & Lefèvre, S. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-54181-5_12 · Cited by 367
Classification of Remote Sensing Images Using EfficientNet-B3 CNN Model With Attention
Alhichri, H., Alswayed, A. S., Bazi, Y., Ammour, N., & Alajlan, N. A. (2021) · IEEE Access · doi:10.1109/access.2021.3051085 · Cited by 357
Wurm, M., Stark, T., Zhu, X. X., Weigand, M., & Taubenböck, H. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.02.006 · Cited by 332
A Feature Difference Convolutional Neural Network-Based Change Detection Method
Zhang, M., & Shi, W. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2981051 · Cited by 320
Li, E., Xia, J., Du, P., Lin, C., & Samat, A. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2711275 · Cited by 296
Cao, H., Shao, H., Zhong, X., Deng, Q., Yang, X., & Xuan, J. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2021.11.016 · Cited by 290
Himeur, Y., Rimal, B., Tiwary, A., & Amira, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.06.003 · Cited by 288
Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling
He, N., Fang, L., Li, S., Plaza, A., & Plaza, J. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2845668 · Cited by 284
Delving deep into spatial pooling for squeeze-and-excitation networks
Jin, X., Xie, Y., Wei, X.-S., Zhao, B.-R., Chen, Z.-M., & Tan, X. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2021.108159 · Cited by 278
Audebert, N., Le Saux, B., & Lefèvre, S. (2017) · Remote Sensing · doi:10.3390/rs9040368 · Cited by 264
A patch-based convolutional neural network for remote sensing image classification
Sharma, A., Liu, X., Yang, X., & Shi, D. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.07.017 · Cited by 260
“Accordingly, the remote sensing community has also started to incorporate deep CNNs to image classification tasks [9, 44, 56, 41, 21, 38].”
BackgroundFully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery
Sherrah, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02585 · Cited by 260
Transfer Learning With Adaptive Fine-Tuning
Vrbancic, G., & Podgorelec, V. (2020) · IEEE Access · doi:10.1109/access.2020.3034343 · Cited by 254
On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID
Long, Y., Xia, G.-S., Li, S., Yang, W., Yang, M. Y., Zhu, X. X., Zhang, L., & Li, D. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3070368 · Cited by 250
MARTA GANs: unsupervised representation learning for remote sensing image classification
Lin, D., Fu, K., Wang, Y., Xu, G., & Sun, X. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2752750 · Cited by 230
“While some of the supervised methods [4], [5] achieved an accuracy above 99%, these methods are fine-tuned from pre-trained models, which are usually trained with a large amount of labeled data (such as ImageNet).”
BackgroundAD-NET: Age-adjust neural network for improved MCI to AD conversion prediction
Gao, F., Yoon, H., Xu, Y., Goradia, D., Luo, J., Wu, T., & Su, Y. (2020) · NeuroImage: Clinical · doi:10.1016/j.nicl.2020.102290 · Cited by 64
Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization
Bashmal, L., Bazi, Y., AlHichri, H., AlRahhal, M., Ammour, N., & Alajlan, N. (2018) · Remote Sensing · doi:10.3390/rs10020351 · Cited by 60
“Another work [30] evaluated and analyzed three strategies using CNN for scene classification, including fully-trained CNN, fine-tuned CNN, and pre-trained CNN used as feature extractors.”
MethodA New Vision-Based Method Using Deep Learning for Damage Inspection in Wind Turbine Blades
Moreno, S., Pena, M., Toledo, A., Trevino, R., & Ponce, H. (2018) · 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE) · doi:10.1109/iceee.2018.8533924 · Cited by 40
“Convolutional neural network (CNN) is a well-known deep learning architecture inspired on the nature of visual perception in living creatures [9] typically applied for classification and regression in image processing [10].”
Background
Metadata
- Authors
- Nogueira, K., Penatti, O. A. B., & dos Santos, J. A.
- Venue
- Pattern Recognition
- Publisher
- Elsevier BV
- Published
- 2017-01
- Volume
- 61
- Pages
- 539-556
- ISSN
- 0031-3203 (ISSN-L)
- OpenAlex
- W2253590344
- Cited by
- 902
- References
- 68
- FWCI
- 115.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.