The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
A deep learning framework for remote sensing image registration
Abstract
We propose an effective deep neural network aiming at remote sensing image registration problem. Unlike conventional methods doing feature extraction and feature matching separately, we pair patches from sensed and reference images, and then learn the mapping directly between these patch-pairs and their matching labels for later registration. This end-to-end architecture allows us to optimize the whole processing (learning mapping function) through information feedback when training the network, which is lacking in conventional methods. In addition, to alleviate the small data issue of remote sensing images for training, our proposal introduces a self-learning by learning the mapping function using images and their transformed copies. Moreover, we apply a transfer learning to reduce the huge computation cost in the training stage. It does not only speed up our framework, but also get extra performance gains. The comprehensive experiments conducted on seven sets of remote sensing images, acquired by Radarsat, SPOT and Landsat, show that our proposal improves the registration accuracy up to 2.4–53.7%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Speeded-Up Robust Features (SURF)
Bay, H., Ess, A., Tuytelaars, T., & Van Gool, L. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.09.014 · Cited by 13,557
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Image registration methods: a survey
Zitová, B., & Flusser, J. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00137-9 · Cited by 6,423
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879
Multimodality image registration by maximization of mutual information
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., & Suetens, P. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.563664 · Cited by 4,547
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Robust wide-baseline stereo from maximally stable extremal regions
Matas, J., Chum, O., Urban, M., & Pajdla, T. (2004) · Image and Vision Computing · doi:10.1016/j.imavis.2004.02.006 · Cited by 3,686
A Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852
Why Does Unsupervised Pre-training Help Deep Learning?
Erhan, D., Bengio, Y., Courville, A. C., Manzagol, P.-A., Vincent, P., & Samy Bengio, A. (2010) · Journal of Machine Learning Research · Cited by 2,631
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
An FFT-based technique for translation, rotation, and scale-invariant image registration
Reddy, B. S., & Chatterji, B. N. (1996) · IEEE Transactions on Image Processing · doi:10.1109/83.506761 · Cited by 1,836
Cheng, G., Zhou, P., & Han, J. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2601622 · Cited by 1,771
ASIFT: A New Framework for Fully Affine Invariant Image Comparison
Morel, J.-M., & Yu, G. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080732730 · Cited by 1,468
Zhao, W., & Du, S. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2543748 · Cited by 1,212
Learning to compare image patches via convolutional neural networks
Zagoruyko, S., & Komodakis, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299064 · Cited by 1,140
MatchNet: Unifying feature and metric learning for patch-based matching
Han, X., Leung, T., Jia, Y., Sukthankar, R., & Berg, A. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298948 · Cited by 825
Discriminative Learning of Deep Convolutional Feature Point Descriptors
Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., & Moreno-Noguer, F. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.22 · Cited by 804
Han, J., Zhang, D., Cheng, G., Guo, L., & Ren, J. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2374218 · Cited by 733
Unsupervised Deep Feature Extraction for Remote Sensing Image Classification
Romero, A., Gatta, C., & Camps-Valls, G. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2478379 · Cited by 729
Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks
Gong, M., Zhao, J., Liu, J., Miao, Q., & Jiao, L. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2435783 · Cited by 638
Self-citationSAR-SIFT: A SIFT-Like Algorithm for SAR Images
Dellinger, F., Delon, J., Gousseau, Y., Michel, J., & Tupin, F. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2323552 · Cited by 634
Liu, J., Gong, M., Qin, K., & Zhang, P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2636227 · Cited by 522
Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity
Ye, Y., Shan, J., Bruzzone, L., & Shen, L. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2656380 · Cited by 487
Learning multiscale and deep representations for classifying remotely sensed imagery
Zhao, W., & Du, S. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.01.004 · Cited by 398
Remote Sensing Image Registration With Modified SIFT and Enhanced Feature Matching
Ma, W., Wen, Z., Wu, Y., Jiao, L., Gong, M., Zheng, Y., & Liu, L. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2016.2600858 · Cited by 389
Self-citationTraining Deep Convolutional Neural Networks for Land–Cover Classification of High-Resolution Imagery
Scott, G. J., England, M. R., Starms, W. A., Marcum, R. A., & Davis, C. H. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2657778 · Cited by 353
Semantic Annotation of High-Resolution Satellite Images via Weakly Supervised Learning
Yao, X., Han, J., Cheng, G., Qian, X., & Guo, L. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2523563 · Cited by 346
A Novel Point-Matching Algorithm Based on Fast Sample Consensus for Image Registration
Wu, Y., Ma, W., Gong, M., Su, L., & Jiao, L. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2325970 · Cited by 342
Self-citationUniform Robust Scale-Invariant Feature Matching for Optical Remote Sensing Images
Sedaghat, A., Mokhtarzade, M., & Ebadi, H. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2144607 · Cited by 334
Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas
Suri, S., & Reinartz, P. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2009.2034842 · Cited by 330
Cheng, G., Han, J., Guo, L., Liu, Z., Bu, S., & Ren, J. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2393857 · Cited by 305
A Novel Coarse-to-Fine Scheme for Automatic Image Registration Based on SIFT and Mutual Information
Gong, M., Zhao, S., Jiao, L., Tian, D., & Wang, S. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2013.2281391 · Cited by 301
Self-citationZhang, P., Gong, M., Su, L., Liu, J., & Li, Z. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.02.013 · Cited by 294
Wu, G., Kim, M., Wang, Q., Munsell, B. C., & Shen, D. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2496253 · Cited by 278
Automatic Image Registration Through Image Segmentation and SIFT
Goncalves, H., Corte-Real, L., & Goncalves, J. A. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2109389 · Cited by 214
Scott, G. J., Marcum, R. A., Davis, C. H., & Nivin, T. W. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2722988 · Cited by 214
BFSIFT: A Novel Method to Find Feature Matches for SAR Image Registration
Wang, S., You, H., & Fu, K. (2012) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2011.2177437 · Cited by 192
Self-citationYe, Y., & Shan, J. (2014) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2014.01.009 · Cited by 186
Cheng, G., Zhou, P., & Han, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.315 · Cited by 175
Fan, B., Huo, C., Pan, C., & Kong, Q. (2013) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2012.2216500 · Cited by 171
Ye, Y., & Shen, L. (2016) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-iii-1-9-2016 · Cited by 155
Applicability of the SIFT operator to geometric SAR image registration
Schwind, P., Suri, S., Reinartz, P. H., & Siebert, A. (2010) · International Journal of Remote Sensing · doi:10.1080/01431160902927622 · Cited by 151
Remote Sensing Image Registration Using Multiple Image Features
Yang, K., Pan, A., Yang, Y., Zhang, S., Ong, S. H., & Tang, H. (2017) · Remote Sensing · doi:10.3390/rs9060581 · Cited by 136
Robust Multispectral Image Registration Using Mutual-Information Models
Kern, J. P., & Pattichis, M. S. (2007) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.892599 · Cited by 119
Ma, X., Wang, H., & Wang, J. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.09.001 · Cited by 114
SAR Image Registration Using Phase Congruency and Nonlinear Diffusion-Based SIFT
Fan, J., Wu, Y., Wang, F., et al. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2351396 · Cited by 102
Fully Automatic Subpixel Image Registration of Multiangle CHRIS/Proba Data
Ma, J., Chan, J. C.-W., & Canters, F. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2042813 · Cited by 101
Zhou, P., Cheng, G., Liu, Z., Bu, S., & Hu, X. (2015) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-015-0370-3 · Cited by 98
Liang, J., Liu, X., Huang, K., Li, X., Wang, D., & Wang, X. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2013.2242895 · Cited by 96
An Efficient SIFT-Based Mode-Seeking Algorithm for Sub-Pixel Registration of Remotely Sensed Images
Kupfer, B., Netanyahu, N. S., & Shimshoni, I. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2343471 · Cited by 95
Multimodal registration of remotely sensed images based on Jeffrey’s divergence
Xu, X., Li, X., Liu, X., Shen, H., & Shi, Q. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.10.005 · Cited by 68
Heo, Y. S., Lee, K. M., & Lee, S. U. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206507 · Cited by 35
Xu, C., Sui, H., Li, D., et al. (2016) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xli-b7-593-2016 · Cited by 5
Research works printed but not linked: 7
- Goncalves, 2009, Measures for an objective evaluation of the geometric correction process quality, IEEE Geosci. Remote Sens. Lett, 6, 292, 10.1109/LGRS.2008.2012441 · doi:10.1109/lgrs.2008.2012441
- Jacqueline, L.M., 2017. Introduction to remote sensing image registration. In: IEEE International Geoscience and Remote Sensing Symposium.
- Krizhevsky, A., Sutskever, I., Hinton, G.E., Dec. 2012. Imagenet classification with deep convolutional neural networks. pp. 1097–1105.
- Parmehr, 2012, Automatic registration of multi-source data using mutual information, ISPRS J. Photogramm. Rem. Sens., I-7, 303
- Patel, M.I., Thakar, V.K., 2015. Speed improvement in image registration using maximum likelihood based mutual information. In: International Conference on Advanced Computing and Communication Systems. (Jan), https://doi.org/10.1109/ICACCS.2015.7324130. · doi:10.1109/icaccs.2015.7324130
- Ye, 2017, Robust optical-to-SAR image matching based on shape properties, IEEE Geosci. Remote Sens. Lett., 14, 564, 10.1109/LGRS.2017.2660067 · DOI
- Yosinski, J., Clune, J., Bengio, Y., Lipson, H., 2014. How transferable are features in deep neural networks? In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (NIPS'14) (December), pp. 3320–3328.
Cited by
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
A review of multimodal image matching: Methods and applications
Jiang, X., Ma, J., Xiao, G., Shao, Z., & Guo, X. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.012 · Cited by 532
SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness
Tang, L., Deng, Y., Ma, Y., Huang, J., & Ma, J. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.106082 · Cited by 488
Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives
Zhu, X. X., Montazeri, S., Ali, M., Hua, Y., Wang, Y., Mou, L., Shi, Y., Xu, F., & Bamler, R. (2021) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2020.3046356 · Cited by 357
Metadata
- Authors
- Wang, S., Quan, D., Liang, X., Ning, M., Guo, Y., & Jiao, L.
- Venue
- ISPRS Journal of Photogrammetry and Remote Sensing
- Publisher
- Elsevier BV
- Published
- 2018-11
- Volume
- 145
- Pages
- 148-164
- ISSN
- 0924-2716 (ISSN-L)
- OpenAlex
- W2781778455
- Cited by
- 292
- References
- 66
- FWCI
- 25.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- National Natural Science Foundation of China (61771379, 61571342)
- Japan Society for the Promotion of Science (15K00236)
- Higher Education Discipline Innovation Project (B07048)
- Program for Changjiang Scholars and Innovative Research Team in University (IRT_15R53)
- National Key Research and Development Program of China (2013CB329402)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.