The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
FusAtNet: Dual Attention based SpectroSpatial Multimodal Fusion Network for Hyperspectral and LiDAR Classification
Open accessAbstract
Abstract With recent advances in sensing, multimodal data is becoming easily available for various applications, especially in remote sensing (RS), where many data types like multispectral imagery (MSI), hyperspectral imagery (HSI), LiDAR etc. are available. Effective fusion of these multisource datasets is becoming important, for these multimodality features have been shown to generate highly accurate land-cover maps. However, fusion in the context of RS is non-trivial considering the redundancy involved in the data and the large domain differences among multiple modalities. In addition, the feature extraction modules for different modalities hardly interact among themselves, which further limits their semantic relatedness. As a remedy, we propose a feature fusion and extraction framework, namely FusAtNet, for collective land-cover classification of HSIs and LiDAR data in this paper. The proposed framework effectively utilizses HSI modality to generate an attention map using "self-attention" mechanism that highlights its own spectral features. Similarly, a "cross-attention" approach is simultaneously used to harness the LiDAR derived attention map that accentuates the spatial features of HSI. These attentive spectral and spatial representations are then explored further along with the original data to obtain modality-specific feature embeddings. The modality oriented joint spectro-spatial information thus obtained, is subsequently utilized to carry out the land-cover classification task. Experimental evaluations on three HSI-LiDAR datasets show that the proposed method achieves the state-of-the-art classification performance, including on the largest HSI-LiDAR dataset available, University of Houston (Data Fusion Contest - 2013), opening new avenues in multimodal feature fusion for classification.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2010) · Cited by 14,134
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · cited 1×
“The streams follow the similar architecture and contains cascaded residual blocks (inspired from Face Alignment Network (FAN) [37] and hourglass networks [38]) to keep both the original and extracted features from fusion.”
Background(2011) · Cited by 2,752
Deep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
Deep Convolutional Neural Networks for Hyperspectral Image Classification
Hu, W., Huang, Y., Wei, L., Zhang, F., & Li, H. (2015) · Journal of Sensors · doi:10.1155/2015/258619 · Cited by 1,941
Incorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,595
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis
Suk, H.-I., Lee, S.-W., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.077 · Cited by 968
A review of remote sensing image fusion methods
Ghassemian, H. (2016) · Information Fusion · doi:10.1016/j.inffus.2016.03.003 · Cited by 849
Hyperspectral Image Classification Using Deep Pixel-Pair Features
Li, W., Wu, G., Zhang, F., & Du, Q. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2616355 · Cited by 805
Multisource Remote Sensing Data Classification Based on Convolutional Neural Network
Xu, X., Li, W., Ran, Q., Du, Q., Gao, L., & Zhang, B. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2756851 · Cited by 689
Local Binary Patterns and Extreme Learning Machine for Hyperspectral Imagery Classification
Li, W., Chen, C., Su, H., & Du, Q. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2381602 · Cited by 685
Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest
Debes, C., Merentitis, A., Heremans, R., Hahn, J., Frangiadakis, N., van Kasteren, T., Liao, W., Bellens, R., Pizurica, A., Gautama, S., et al. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2305441 · Cited by 653
Fusion of Hyperspectral and LIDAR Remote Sensing Data for Classification of Complex Forest Areas
Dalponte, M., Bruzzone, L., & Gianelle, D. (2008) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2008.916480 · Cited by 564
Multimodal Classification of Remote Sensing Images: A Review and Future Directions
Gomez-Chova, L., Tuia, D., Moser, G., & Camps-Valls, G. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2449668 · Cited by 508
Deep Learning-Based Image Segmentation on Multimodal Medical Imaging
Guo, Z., Li, X., Huang, H., Guo, N., & Li, Q. (2019) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2018.2890359 · Cited by 440
Jetley, S., Lord, N. A., Lee, N., & Torr, P. H. S. (2018) · arXiv · doi:10.48550/arxiv.1804.02391 · Cited by 383 · cited 1×
“Recently, the usage of attention learning mechanism has shown remarkable performance gain for different visual inference tasks [18, 19, 20].”
BackgroundFeature Extraction for Classification of Hyperspectral and LiDAR Data Using Patch-to-Patch CNN
Zhang, M., Li, W., Du, Q., Gao, L., & Zhang, B. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2864670 · Cited by 292
Support vector machines for hyperspectral image classification with spectral-based kernels
Mercier, G., & Lennon, M. (2004) · IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477) · doi:10.1109/igarss.2003.1293752 · Cited by 275
Visual Attention-Driven Hyperspectral Image Classification
Haut, J. M., Paoletti, M. E., Plaza, J., Plaza, A., & Li, J. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2918080 · Cited by 251
Deep Fusion of Remote Sensing Data for Accurate Classification
Chen, Y., Li, C., Ghamisi, P., Jia, X., & Gu, Y. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2704625 · Cited by 245
Mou, L., & Zhu, X. X. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2933609 · Cited by 233
Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach
Liang, M., Li, Z., Chen, T., & Zeng, J. (2015) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2014.2377729 · Cited by 231
Nearest Regularized Subspace for Hyperspectral Classification
Li, W., Tramel, E. W., Prasad, S., & Fowler, J. E. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2013.2241773 · Cited by 227
Multimodal Deep Learning for Cervical Dysplasia Diagnosis
Xu, T., Zhang, H., Huang, X., Zhang, S., & Metaxas, D. N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_14 · Cited by 187
Koetz, B., Morsdorf, F., van der Linden, S., Curt, T., & Allgöwer, B. (2008) · Forest Ecology and Management · doi:10.1016/j.foreco.2008.04.025 · Cited by 186
Survey of Deep-Learning Approaches for Remote Sensing Observation Enhancement
Tsagkatakis, G., Aidini, A., Fotiadou, K., Giannopoulos, M., Pentari, A., & Tsakalides, P. (2019) · Sensors · doi:10.3390/s19183929 · Cited by 157
Fusion of Hyperspectral and LiDAR Data Using Sparse and Low-Rank Component Analysis
Rasti, B., Ghamisi, P., Plaza, J., & Plaza, A. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2726901 · Cited by 151
Feng, Q., Zhu, D., Yang, J., & Li, B. (2019) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi8010028 · Cited by 116
Remote Sensing Image Fusion Based on Adaptive IHS and Multiscale Guided Filter
Yang, Y., Wan, W., Huang, S., Yuan, F., Yang, S., & Que, Y. (2016) · IEEE Access · doi:10.1109/access.2016.2599403 · Cited by 78
CMIR-NET : A deep learning based model for cross-modal retrieval in remote sensing
Chaudhuri, U., Banerjee, B., Bhattacharya, A., et al. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.02.006 · Cited by 65
Self-citationEslami, M., & Mohammadzadeh, A. (2015) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2489838 · Cited by 60
Morchhale, S., Pauca, V. P., Plemmons, R. J., & Torgersen, T. C. (2016) · Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing · doi:10.1109/whispers.2016.8071715 · Cited by 59
Supervised spatial classification of multispectral LiDAR data in urban areas
Huo, L., Silva, C. A., Klauberg, C., Mohan, M., Zhao, L., Tang, P. T. P., & Hudak, A. T. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206185 · Cited by 53
Multimodal learning and inference from visual and remotely sensed data
Rao, D., Deuge, M. D., Nourani‐Vatani, N., Williams, S. B., & Pizarro, O. R. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364916679892 · Cited by 47
Zhao, C., Gao, X., Wang, Y., & Li, J. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2535538 · Cited by 46
Deep fusion of hyperspectral and LiDAR data for thematic classification
Chen, Y., Li, C., Ghamisi, P., et al. (2016) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2016.7729930 · Cited by 28
Multiple Sources Data Fusion Via Deep Forest
Xia, J., Ming, Z., & Iwasaki, A. (2018) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2018.8517679 · Cited by 16
Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classification and Beyond
Wang, A., Cai, J., Lu, J., & Cham, T. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3115932 · Cited by 11
Cited by
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Artificial Intelligence for Remote Sensing Data Analysis: A review of challenges and opportunities
Zhang, L., & Zhang, L. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3145854 · Cited by 604
Zhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480
Omia, E., Bae, H., Park, E., Kim, M. S., Baek, I., Kabenge, I., & Cho, B.-K. (2023) · Remote Sensing · doi:10.3390/rs15020354 · Cited by 301
A Feature Selection Method for Multimodal Multispectral LiDAR Sensing
Han, Y., Salido-Monzú, D., Butt, J. A., Schweizer, S., & Wieser, A. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.04.022 · Cited by 6
Metadata
- Authors
- Mohla, S., Pande, S., Banerjee, B., & Chaudhuri, S.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 416-425
- OpenAlex
- W3035356612
- Cited by
- 276
- References
- 43
- FWCI
- 52.52
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.