Exploiting Deep Matching and SAR Data for the Geo-Localization Accuracy Improvement of Optical Satellite Images
Open accessThe DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Improving the geo-localization of optical satellite images is an important pre-processing step for many remote sensing tasks like monitoring by image time series or scene analysis after sudden events. These tasks require geo-referenced and precisely co-registered multi-sensor data. Images captured by the high resolution synthetic aperture radar (SAR) satellite TerraSAR-X exhibit an absolute geo-location accuracy within a few decimeters. These images represent therefore a reliable source to improve the geo-location accuracy of optical images, which is in the order of tens of meters. In this paper, a deep learning-based approach for the geo-localization accuracy improvement of optical satellite images through SAR reference data is investigated. Image registration between SAR and optical images requires few, but accurate and reliable matching points. These are derived from a Siamese neural network. The network is trained using TerraSAR-X and PRISM image pairs covering greater urban areas spread over Europe, in order to learn the two-dimensional spatial shifts between optical and SAR image patches. Results confirm that accurate and reliable matching points can be generated with higher matching accuracy and precision with respect to state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the image registration methods using deep learning are based on a Siamese network (Bromley et al., 1994;Merkle et al., 2017;He et al., 2018;Hughes et al., 2018a;Wang et al., 2018b).”
From Deep learning in remote sensing applications: A meta-analysis and review (2019) · cites this paper as Merkle et al., 2017
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 2×
“In contrast, dilated convolutions [46] systematically aggregate information through an exponential growth of the receptive without degradation in resolution.”
BackgroundMethodA Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Deep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Pansharpening by Convolutional Neural Networks
Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016) · Remote Sensing · doi:10.3390/rs8070594 · Cited by 1,152
Learning to compare image patches via convolutional neural networks
Zagoruyko, S., & Komodakis, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299064 · Cited by 1,140
DeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.175 · Cited by 997
Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches
Žbontar, J., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.05970 · Cited by 931
MatchNet: Unifying feature and metric learning for patch-based matching
Han, X., Leung, T., Jia, Y., Sukthankar, R., & Berg, A. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298948 · Cited by 825
Iterative Weighted Maximum Likelihood Denoising With Probabilistic Patch-Based Weights
Deledalle, C.-A., Denis, L., & Tupin, F. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2029593 · Cited by 823
Discriminative Learning of Deep Convolutional Feature Point Descriptors
Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., & Moreno-Noguer, F. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.22 · Cited by 804
Efficient Deep Learning for Stereo Matching
Luo, W., Schwing, A. G., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.614 · Cited by 741
Self-citationLearning to Detect Roads in High-Resolution Aerial Images
Mnih, V., & Hinton, G. E. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_16 · Cited by 624
An Introduction to Restricted Boltzmann Machines
Fischer, A., & Igel, C. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33275-3_2 · Cited by 569
A contour-based approach to multisensor image registration
Li, H., Manjunath, B. S., & Mitra, S. K. (1995) · IEEE Transactions on Image Processing · doi:10.1109/83.366480 · Cited by 485
Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking
Wang, Q., Lin, J., & Yuan, Y. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2477537 · Cited by 483
Learning deep representations for ground-to-aerial geolocalization
Lin, T.-Y., Cui, Y., Belongie, S., & Hays, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299135 · Cited by 401
Mutual-Information-Based Registration of TerraSAR-X and Ikonos Imagery in Urban Areas
Suri, S., & Reinartz, P. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2009.2034842 · Cited by 330
The TerraSAR-X Mission and System Design
Werninghaus, R., & Buckreuss, S. (2010) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2009.2031062 · Cited by 324
High-Resolution SAR Image Classification via Deep Convolutional Autoencoders
Geng, J., Fan, J., Wang, H., Ma, X., Li, B., & Chen, F. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2478256 · Cited by 291
Liang, H., & Li, Q. (2016) · Remote Sensing · doi:10.3390/rs8020099 · Cited by 238
On the possibility of automatic multisensor image registration
Inglada, J., & Giros, A. (2004) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2004.835294 · Cited by 215
Imaging Geodesy—Toward Centimeter-Level Ranging Accuracy With TerraSAR-X
Eineder, M., Minet, C., Steigenberger, P., Cong, X., & Fritz, T. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2060264 · Cited by 210
Principles of Digital Image Processing: Core Algorithms
Burger, W., & Burge, M. J. (2009) · Undergraduate Topics in Computer Science · doi:10.1007/978-1-84800-195-4 · Cited by 191
Fan, B., Huo, C., Pan, C., & Kong, Q. (2013) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2012.2216500 · Cited by 171
Learning to Compare Image Patches via Convolutional Neural Networks
Zagoruyko, S., & Nikos Komodakis, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.03641 · Cited by 159
Ye, Y., & Shen, L. (2016) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-iii-1-9-2016 · Cited by 155
Estimation of Mutual Information: A Survey
Walters-Williams, J., & Li, Y. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02962-2_49 · Cited by 145
HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images
Mattyus, G., Wang, S., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.393 · Cited by 139
Self-citationAn improved model for automatic feature-based registration of SAR and SPOT images
Dare, P. M., & Dowman, I. J. (2001) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/s0924-2716(01)00031-4 · Cited by 137
Exploiting Semantic Information and Deep Matching for Optical Flow
Bai, M., Luo, W., Kundu, K., & Urtasun, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_10 · Cited by 125
Self-citationSui, H., Xu, C., Liu, J., & Hua, F. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2431498 · Cited by 89
Learning to Match Aerial Images with Deep Attentive Architectures
Altwaijry, H., Trulls, E., Hays, J., Fua, P., & Belongie, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.385 · Cited by 84
Learning to Detect and Match Keypoints with Deep Architectures
Altwaijry, H., Veit, A., & Belongie, S. (2016) · British Machine Vision Conference (BMVC) · doi:10.5244/c.30.49 · Cited by 79
Reinartz, P. H., Müller, R., Schwind, P., Suri, S., & Bamler, R. (2010) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2010.10.003 · Cited by 70
Self-citationAutomated Georeferencing of Optical Satellite Data with Integrated Sensor Model Improvement
Müller, R., Krauß, T., Schneider, M., & Reinartz, P. H. (2012) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.78.1.61 · Cited by 57
Self-citationA Robust Technique for Precise Registration of Radar and Optical Satellite Images
Hong, T. D., & Schowengerdt, R. A. (2005) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.71.5.585 · Cited by 53
Hasan, M. M., Pickering, M. R., & Jia, X. (2012) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2012.2187456 · Cited by 47
A visual circle based image registration algorithm for optical and SAR imagery
Shi, W., Su, F., Wang, R., & Fan, J. (2012) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2012.6351089 · Cited by 29
Xu, C., Sui, H., Li, H., & Liu, J. (2015) · International Journal of Remote Sensing · doi:10.1080/01431161.2015.1070321 · Cited by 26
A generic framework for image rectification using multiple types of feature
Long, T., Jiao, W., He, G., Zhang, Z., Cheng, B., & Wang, W. (2015) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2015.01.015 · Cited by 23
Automatic registration of SAR and optical images based on mutual information assisted Monte Carlo
Siddique, M. A., Sarfraz, M. S., Bornemann, D., & Hellwich, O. (2012) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2012.6351159 · Cited by 21
Multi-modal image matching based on local frequency information
Liu, X., Lei, Z., Yu, Q., Zhang, X., Shang, Y., & Hou, W. (2013) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/1687-6180-2013-3 · Cited by 21
Matching Between SAR Images and Optical Images based on HOG descriptor
Li, Q., Li, Z., & Qu, G. (2013) · IET International Radar Conference · doi:10.1049/cp.2013.0405 · Cited by 20
Multisource data registration based on NURBS description of contours
Pan, C., Zhang, Z., Yan, H., Wu, G., & Ma, S. (2007) · International Journal of Remote Sensing · doi:10.1080/01431160701294687 · Cited by 20
Linear Signatures in Urban SAR Images—Partly Misinterpreted?
Auer, S., & Gernhardt, S. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2308353 · Cited by 19
Self-citationAutomatic and accurate registration of VHR optical and SAR images using a quadtree structure
Han, Y., & Byun, Y.-G. (2015) · International Journal of Remote Sensing · doi:10.1080/01431161.2015.1030046 · Cited by 16
Using worldwide available TerraSAR-X data to calibrate the geo-location accuracy of optical sensors
Perko, R., Raggam, H., Gutjahr, K., & Schardt, M. (2011) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2011.6049732 · Cited by 14
Schneider, M., Müller, R., Krauß, T., Reinartz, P. H., Hörsch, B., & Schmuck, S. (2010) · elib (German Aerospace Center) · Cited by 7
Self-citationREGISTRATION OF OPTICAL AND SAR SATELLITE IMAGES BASED ON GEOMETRIC FEATURE TEMPLATES
Merkle, N., Müller, R. C., & Reinartz, P. H. (2015) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprsarchives-xl-1-w5-447-2015 · Cited by 7
Self-citation
Cited by
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456
Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives
Zhu, X. X., Montazeri, S., Ali, M., Hua, Y., Wang, Y., Mou, L., Shi, Y., Xu, F., & Bamler, R. (2021) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2020.3046356 · Cited by 357
THE SEN1-2 DATASET FOR DEEP LEARNING IN SAR-OPTICAL DATA FUSION
Schmitt, M., Hughes, L. H., & Zhu, X. X. (2018) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprs-annals-iv-1-141-2018 · Cited by 327
Pixel level fusion techniques for SAR and optical images: A review
Kulkarni, S. C., & Rege, P. P. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.003 · Cited by 318
LNIFT: Locally Normalized Image for Rotation Invariant Multimodal Feature Matching
Li, J., Xu, W., Shi, P., Zhang, Y., & Hu, Q. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3165940 · Cited by 193
Metadata
- Authors
- Merkle, N., Luo, W., Auer, S., Müller, R., & Urtasun, R.
- Affiliations
- Deutsches Zentrum für Luft- und Raumfahrt e. V. (DLR) (DE)
- University of Toronto (CA)
- Corresponding author
- Merkle, Nina
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2017-06-10
- Volume
- 9
- Issue
- 6
- Pages
- 586
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2608922915
- Cited by
- 120
- References
- 55
- Citations by year
- 2026: 6 · 2025: 6 · 2024: 17 · 2023: 15 · 2022: 18 · 2021: 21 · 2020: 16 · 2019: 10 · 2018: 11
- FWCI
- 8.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Remote Sensing (publishedVersion, cc-by)
- elib (German Aerospace Center) (acceptedVersion)
- TSpace (University of Toronto) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Satellite Image Processing and Photogrammetry
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Keywords
- geolocation, optical satellite imagery, synthetic aperture radar, TerraSAR-X, PRISM, image registration, Siamese network, urban environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.