Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images
Abstract
Object detection in very high resolution optical remote sensing images is a fundamental problem faced for remote sensing image analysis. Due to the advances of powerful feature representations, machine-learning-based object detection is receiving increasing attention. Although numerous feature representations exist, most of them are handcrafted or shallow-learning-based features. As the object detection task becomes more challenging, their description capability becomes limited or even impoverished. More recently, deep learning algorithms, especially convolutional neural networks (CNNs), have shown their much stronger feature representation power in computer vision. Despite the progress made in nature scene images, it is problematic to directly use the CNN feature for object detection in optical remote sensing images because it is difficult to effectively deal with the problem of object rotation variations. To address this problem, this paper proposes a novel and effective approach to learn a rotation-invariant CNN (RICNN) model for advancing the performance of object detection, which is achieved by introducing and learning a new rotation-invariant layer on the basis of the existing CNN architectures. However, different from the training of traditional CNN models that only optimizes the multinomial logistic regression objective, our RICNN model is trained by optimizing a new objective function via imposing a regularization constraint, which explicitly enforces the feature representations of the training samples before and after rotating to be mapped close to each other, hence achieving rotation invariance. To facilitate training, we first train the rotation-invariant layer and then domain-specifically fine-tune the whole RICNN network to further boost the performance. Comprehensive evaluations on a publicly available ten-class object detection data set demonstrate the effectiveness of the proposed method.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Designing rotation invariant loss functions is a recent popular solution, where a constraint on the detection loss is added so that the feature of rotated objects keeps unchanged.
“Designing rotation invariant loss functions is a recent popular solution, where a constraint on the detection loss is added so that the feature of rotated objects keeps unchanged [169-171].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [170]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · see citing works
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · see citing works
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980 · see citing works
A Bayesian Hierarchical Model for Learning Natural Scene Categories
Fei-Fei, L., & Perona, P. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.16 · Cited by 3,657 · see citing works
Region-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714 · see citing works
A survey on object detection in optical remote sensing images
Cheng, G., & Han, J. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.03.014 · Cited by 1,503 · see citing works
Self-citationHu, F., Xia, G.-S., Hu, J., & Zhang, L. (2015) · Remote Sensing · doi:10.3390/rs71114680 · Cited by 1,199 · see citing works
Cheng, G., Han, J., Zhou, P., & Guo, L. (2014) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2014.10.002 · Cited by 1,007 · see citing works
Self-citationHan, J., Zhang, D., Cheng, G., Guo, L., & Ren, J. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2374218 · Cited by 734 · see citing works
Self-citationBackground Prior-Based Salient Object Detection via Deep Reconstruction Residual
Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., & Wu, F. (2014) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2381471 · Cited by 445 · see citing works
Self-citationSparse Representation for Target Detection in Hyperspectral Imagery
Chen, Y., Nasrabadi, N. M., & Tran, T. D. (2011) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2011.2113170 · Cited by 431 · see citing works
Zhu, C., Zhou, H., Wang, R., & Guo, J. H. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2046330 · Cited by 403 · see citing works
Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning
Yao, X., Han, J., Cheng, G., Qian, X., & Guo, L. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2523563 · Cited by 346 · see citing works
Self-citationCheng, G., Han, J., Guo, L., Liu, Z., Bu, S., & Ren, J. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2393857 · Cited by 305 · see citing works
Self-citationCheng, G., Guo, L., Zhao, T., Han, J., Li, H., & Fang, J. (2012) · International Journal of Remote Sensing · doi:10.1080/01431161.2012.705443 · Cited by 281 · see citing works
Self-citationDas, S., Mirnalinee, T. T., & Varghese, K. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2136381 · Cited by 222 · see citing works
Sun, H., Sun, X., Wang, H., Li, Y., & Li, X. (2011) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2011.2161569 · Cited by 219 · see citing works
Han, J., Zhou, P., Zhang, D., Cheng, G., Guo, L., Liu, Z., Bu, S., & Wu, J. (2014) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2013.12.011 · Cited by 200 · see citing works
Self-citationA Sparse Representation-Based Binary Hypothesis Model for Target Detection in Hyperspectral Images
Zhang, Y., Du, B., & Zhang, L. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2337883 · Cited by 198 · see citing works
Sparse Transfer Manifold Embedding for Hyperspectral Target Detection
Zhang, L., Zhang, L., Tao, D., & Huang, X. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2013.2246837 · Cited by 187 · see citing works
Zhong, P., & Wang, R. (2007) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.907109 · Cited by 180 · see citing works
Object Classification of Aerial Images With Bag-of-Visual Words
Xu, S., Fang, T., Li, D., & Wang, S. (2009) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2009.2035644 · Cited by 174 · see citing works
Airborne Vehicle Detection in Dense Urban Areas Using HoG Features and Disparity Maps
Tuermer, S., Kurz, F., Reinartz, P. H., & Stilla, U. (2013) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2013.2242846 · Cited by 169 · see citing works
Object detection in remote sensing imagery using a discriminatively trained mixture model
Cheng, G., Han, J., Guo, L., Qian, X., Zhou, P., Yao, X., & Hu, X. (2013) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2013.08.001 · Cited by 168 · see citing works
Self-citationVehicle Detection in Very High Resolution Satellite Images of City Areas
Jens Leitloff, A., Hinz, S., & Stilla, U. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2043109 · Cited by 168 · see citing works
Hu, F., Xia, G.-S., Wang, Z., Huang, X., Zhang, L., & Sun, H. (2015) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2444405 · Cited by 164 · see citing works
Huang, X., & Zhang, L.-P. (2009) · International Journal of Remote Sensing · doi:10.1080/01431160802546837 · Cited by 154 · see citing works
On-line boosting-based car detection from aerial images
Gräbner, H., Thủy, N. T. T., Gruber, B., & Bischof, H. (2008) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2007.10.005 · Cited by 146 · see citing works
Two-Stage Learning to Predict Human Eye Fixations via SDAEs
Han, J., Zhang, D., Wen, S., Guo, L., Liu, T., & Li, X. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2404432 · Cited by 139 · see citing works
Self-citationClassification-based vehicle detection in high-resolution satellite images
Eikvil, L., Aurdal, L., & Koren, H. (2008) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2008.09.005 · Cited by 129 · see citing works
Non-Local Sparse Unmixing for Hyperspectral Remote Sensing Imagery
Zhong, Y., Feng, R., & Zhang, L. (2013) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2013.2280063 · Cited by 128 · see citing works
A Visual Search Inspired Computational Model for Ship Detection in Optical Satellite Images
Bi, F., Zhu, B., Gao, L., & Bian, M. (2012) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2011.2180695 · Cited by 122 · see citing works
Accurate Junction Detection and Characterization in Natural Images
Xia, G.-S., Delon, J., & Gousseau, Y. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0640-1 · Cited by 121 · see citing works
Texture-Based Airport Runway Detection
Aytekın, Ö., Zongur, U., & Halıcı, U. (2012) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2012.2210189 · Cited by 119 · see citing works
Weakly Supervised Learning for Target Detection in Remote Sensing Images
Zhang, D., Han, J., Cheng, G., Liu, Z., Bu, S., & Guo, L. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2358994 · Cited by 108 · see citing works
Self-citationShape-based Invariant Texture Indexing
Xia, G.-S., Delon, J., & Gousseau, Y. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0312-3 · Cited by 100 · see citing works
Zhou, P., Cheng, G., Liu, Z., Bu, S., & Hu, X. (2015) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-015-0370-3 · Cited by 95 · see citing works
Self-citationYao, X., Han, J., Guo, L., Bu, S., & Liu, Z. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.073 · Cited by 94 · see citing works
Self-citationObject Detection Based on Sparse Representation and Hough Voting for Optical Remote Sensing Imagery
Yokoya, N., & Iwasaki, A. (2015) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2404578 · Cited by 90 · see citing works
Cheng, G., Zhou, P., Han, J., Guo, L., & Han, J. (2015) · IET Computer Vision · doi:10.1049/iet-cvi.2014.0270 · Cited by 66 · see citing works
Self-citationLearning coarse-to-fine sparselets for efficient object detection and scene classification
Cheng, G., Han, J., Guo, L., & Liu, T. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298721 · Cited by 61 · see citing works
Self-citationOne-Class Classification of Remote Sensing Images Using Kernel Sparse Representation
Song, B., Li, P., Li, J., et al. (2016) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2508285 · Cited by 51 · see citing works
Xia, G.-S., Wang, Z., Xiong, C., et al. (2015) · Remote Sensing · doi:10.3390/rs71115014 · Cited by 50 · see citing works
A Nonlinear Sparse Representation-Based Binary Hypothesis Model for Hyperspectral Target Detection
Zhang, Y., Zhang, L., Du, B., et al. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2368173 · Cited by 44 · see citing works
Yao, C., & Cheng, G. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.05.017 · Cited by 23 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · see citing works
“After the great success of RCNN in 2014, deep CNN has been soon applied to remote sensing target detection [275, 276, 400, 401].”
BackgroundMethodDOTA: A Large-Scale Dataset for Object Detection in Aerial Images
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00418 · Cited by 3,414 · see citing works
Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762307 · Cited by 3,400 · see citing works
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456 · see citing works
Object detection in optical remote sensing images: A survey and a new benchmark
Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.023 · Cited by 2,317 · see citing works
Self-citationLearning RoI Transformer for Oriented Object Detection in Aerial Images
Ding, J., Xue, N., Long, Y., Xia, G.-S., & Lu, Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00296 · Cited by 1,666 · see citing works
A survey and performance evaluation of deep learning methods for small object detection
Liu, Y., Sun, P., Wergeles, N., & Shang, Y. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114602 · Cited by 665 · see citing works
Ball, J. E., Anderson, D. T., & Chan, C. S. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.042609 · Cited by 746 · see citing works
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584 · see citing works
Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning
Kemker, R., Salvaggio, C., & Kanan, C. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.04.014 · Cited by 558 · see citing works
PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval
Zhou, W., Newsam, S., Li, C., & Shao, Z. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.01.004 · Cited by 571 · see citing works
Why do deep convolutional networks generalize so poorly to small image transformations?
Azulay, A., & Weiss, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.12177 · Cited by 428 · see citing works
“Alternatively, one could use specially designed features such as SIFT in which invariance is hard coded [7, 29, 30, 31] or neural network architectures that explicitly enforce invariance [36, 19, 5, 6, 11, 12, 47, 46, 8].”
BackgroundMethodCheng, G., Han, J., Zhou, P., & Xu, D. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2867198 · Cited by 408 · see citing works
Self-citationRingMo: A Remote Sensing Foundation Model With Masked Image Modeling
Sun, X., Wang, P., Lu, W., Zhu, Z., Lu, X., He, Q., Li, J., Rong, X., Yang, Z., Chang, H., et al. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3194732 · Cited by 323 · see citing works
Deng, Z., Sun, H., Zhou, S., Zhao, J., & Zou, H. (2017) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2017.2694890 · Cited by 195 · see citing works
Yan, J., Wang, H., Yan, M., Diao, W., Sun, X., & Li, H. (2019) · Remote Sensing · doi:10.3390/rs11030286 · Cited by 147 · see citing works
Tayara, H., & Chong, K. T. (2018) · Sensors · doi:10.3390/s18103341 · Cited by 117 · see citing works
Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis
Li, X., Hu, X., Qi, X., Yu, L., Zhao, W., Heng, P.-A., & Xing, L. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3075244 · Cited by 105 · see citing works
SpaceNet MVOI: A Multi-View Overhead Imagery Dataset
Weir, N., Lindenbaum, D., Bastidas, A., Etten, A., Kumar, V., Mcpherson, S., Shermeyer, J., & Tang, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00108 · Cited by 95 · see citing works
Shamsolmoali, P., Chanussot, J., Zareapoor, M., Zhou, H., & Yang, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3106442 · Cited by 44 · see citing works
Online Exemplar-Based Fully Convolutional Network for Aircraft Detection in Remote Sensing Images
Cai, B., Jiang, Z., Zhang, H., Yao, Y., & Nie, S. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2829147 · Cited by 26 · see citing works
- Authors
- Cheng, G., Zhou, P., & Han, J.
- Affiliations
- Northwestern Polytechnical University (CN)
- Venue
- IEEE Transactions on Geoscience and Remote Sensing
- Published
- 2016-12
- Volume
- 54
- Issue
- 12
- Pages
- 7405-7415
- ISSN
- 0196-2892 (ISSN-L)
- 1558-0644
- OpenAlex
- W2512351403
- Cited by
- 1,771
- References
- 52
- Citations by year
- 2026: 87 · 2025: 158 · 2024: 153 · 2023: 176 · 2022: 192 · 2021: 227 · 2020: 257 · 2019: 268 · 2018: 138 · 2017: 113 · 2016: 2
- FWCI
- 187.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Geoscience and Remote Sensing (publishedVersion)
- Funders
- National Science Foundation of China (61401357, 61473231)
- Fundamental Research Funds for the Central Universities (3102016ZY023)
- Innovation Foundation for Doctor Dissertation of NPU (CX201622)
- Aerospace Science Foundation of China (20140153003)
- Topics
- Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- object detection, rotation invariance, fine-tuning, feature representation, remote sensing image analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
