The DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Multimodal Classification of Remote Sensing Images: A Review and Future Directions
Open accessAbstract
Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images. We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image classification; 4) multisensor image fusion exploiting physically-based feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 2×
“The general problem depicted here is also known as domain adaptation [61].”
BackgroundA tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 1×
“This Markovianity condition, which extends the analogous well-known assumption for 1-D Markov chains [135] to 2-D stochastic processes over discrete lattices, is aimed at formalizing spatial–contextual information (spatial memory) associated with the process.”
BackgroundStochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Remote Sensing and Image Interpretation
Wright, J., Lillesand, T. M., & Kiefer, R. W. (1980) · The Geographical Journal · doi:10.2307/634969 · Cited by 7,238
Image Super-Resolution Via Sparse Representation
Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2050625 · Cited by 5,217
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services
Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinti, P., Martimort, P., et al. (2012) · Remote Sensing of Environment · doi:10.1016/j.rse.2011.11.026 · Cited by 4,815
Cited by
Deep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673
Self-citationDeep Learning in Remote Sensing: A Comprehensive Review and List of Resources
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762307 · Cited by 3,400
Self-citationChen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2584107 · Cited by 2,984
Remote Sensing Image Scene Classification: Benchmark and State of the Art
Cheng, G., Han, J., & Lu, X. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2675998 · Cited by 2,926
More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification
Hong, D., Gao, L., Yokoya, N., Yao, J., Chanussot, J., Du, Q., & Zhang, B. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3016820 · Cited by 1,374
Cheng, G., Xie, X., Han, J., Guo, L., & Xia, G.-S. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3005403 · Cited by 995
Domain adaptation for the classification of remote sensing data: An overview of recent advances
Tuia, D., Persello, C., & Bruzzone, L. (2016) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2548504 · Cited by 675
Self-citationAdvanced Spectral Classifiers for Hyperspectral Images: A review
Ghamisi, P., Plaza, J., Chen, Y., Li, J., & Plaza, A. J. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2616418 · Cited by 675
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Scene Classification With Recurrent Attention of VHR Remote Sensing Images
Wang, Q., Liu, S., Chanussot, J., & Li, X. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2864987 · Cited by 613
Remote Sensing and Cropping Practices: A Review
Bégué, A., Arvor, D., Bellon, B., Betbeder, J., De Abelleyra, D., P. D. Ferraz, R., Lebourgeois, V., Lelong, C., Simões, M., & R. Verón, S. (2018) · Remote Sensing · doi:10.3390/rs10010099 · Cited by 465
Deep Hyperspectral Image Sharpening
Dian, R., Li, S., Guo, A., & Fang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2798162 · Cited by 412
A review of neural networks in plant disease detection using hyperspectral data
Golhani, K., Balasundram, S. K., Vadamalai, G., & Pradhan, B. (2018) · Information Processing in Agriculture · doi:10.1016/j.inpa.2018.05.002 · Cited by 402
Yao, J., Zhang, B., Li, C., Hong, D., & Chanussot, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3284671 · Cited by 390
SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagery
Zhang, J., Lei, J., Xie, W., Fang, Z., Li, Y., & Du, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3258666 · Cited by 388
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Dian, R., Li, S., & Kang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980398 · Cited by 328
Pixel level fusion techniques for SAR and optical images: A review
Kulkarni, S. C., & Rege, P. P. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.003 · Cited by 318
Li, X., Zhang, G., Cui, H., Hou, S., Wang, S., Li, X., Chen, Y., Li, Z., & Zhang, L. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2021.102638 · Cited by 303
Mohla, S., Pande, S., Banerjee, B., & Chaudhuri, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00054 · Cited by 276
Hyperspectral Images Classification With Gabor Filtering and Convolutional Neural Network
Chen, Y., Zhu, L., Ghamisi, P., Jia, X., Li, G., & Tang, L. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2764915 · Cited by 270
Recent advances and new guidelines on hyperspectral and multispectral image fusion
Dian, R., Li, S., Sun, B., & Guo, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.11.001 · Cited by 268
X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data
Hong, D., Yokoya, N., Xia, G.-S., Chanussot, J., & Zhu, X. X. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.06.014 · Cited by 257
Missing Modalities Imputation via Cascaded Residual Autoencoder
Tran, L., Liu, X., Zhou, J., & Jin, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.528 · Cited by 256
Systematic review of deep learning techniques in plant disease detection
Nagaraju, M., & Chawla, P. (2020) · International Journal of System Assurance Engineering and Management · doi:10.1007/s13198-020-00972-1 · Cited by 255
Machine Learning Methods for Weather Forecasting: A Survey
Zhang, H., Liu, Y., Zhang, C., & Li, N. (2025) · Atmosphere · doi:10.3390/atmos16010082 · Cited by 113
Metadata
- Authors
- Gomez-Chova, L., Tuia, D., Moser, G., & Camps-Valls, G.
- Venue
- Proceedings of the IEEE
- Published
- 2015-09
- Volume
- 103
- Issue
- 9
- Pages
- 1560-1584
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W1584663654
- Cited by
- 508
- References
- 107
- FWCI
- 52.46
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.