The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
TTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution
Abstract
Transformer-based method has demonstrated promising performance in image super-resolution tasks, due to its long-range and global aggregation capability. However, the existing Transformer brings two critical challenges for applying it in large-area earth observation scenes: (1) redundant token representation due to most irrelevant tokens; (2) single-scale representation which ignores scale correlation modeling of similar ground observation targets. To this end, this paper proposes to adaptively eliminate the interference of irreverent tokens for a more compact self-attention calculation. Specifically, we devise a Residual Token Selective Group (RTSG) to grasp the most crucial token by dynamically selecting the top- k keys in terms of score ranking for each query. For better feature aggregation, a Multi-scale Feed-forward Layer (MFL) is developed to generate an enriched representation of multi-scale feature mixtures during feed-forward process. Moreover, we also proposed a Global Context Attention (GCA) to fully explore the most informative components, thus introducing more inductive bias to the RTSG for an accurate reconstruction. In particular, multiple cascaded RTSGs form our final Top- k Token Selective Transformer (TTST) to achieve progressive representation. Extensive experiments on simulated and real-world remote sensing datasets demonstrate our TTST could perform favorably against state-of-the-art CNN-based and Transformer-based methods, both qualitatively and quantitatively. In brief, TTST outperforms the state-of-the-art approach (HAT-L) in terms of PSNR by 0.14 dB on average, but only accounts for 47.26% and 46.97% of its computational cost and parameters. The code and pre-trained TTST will be available at https://github.com/XY-boy/TTST for validation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
Enhanced Deep Residual Networks for Single Image Super-Resolution
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.151 · Cited by 7,736
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
Restormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
DOTA: A Large-Scale Dataset for Object Detection in Aerial Images
Xia, G.-S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00418 · Cited by 3,414
Self-citationLi, X., Wang, W., Hu, X., & Yang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00060 · Cited by 3,134
Remote Sensing Image Scene Classification: Benchmark and State of the Art
Cheng, G., Han, J., & Lu, X. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2675998 · Cited by 2,926
Example-based super-resolution
Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002) · IEEE Computer Graphics and Applications · doi:10.1109/38.988747 · Cited by 2,498
Vision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · cited 1×
“Recent studies [55], [56], [57] have demonstrated the effectiveness of large kernel convolutions in improving the performance of vision tasks.”
BackgroundAID: A benchmark data set for performance evaluation of aerial scene classification
Xia, G.-S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., & Lu, X. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2685945 · Cited by 2,372
Self-citationObject detection in optical remote sensing images: A survey and a new benchmark
Li, K., Wan, G., Cheng, G., Meng, L., & Han, J. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.11.023 · Cited by 2,317 · cited 1×
“0 [64], DIOR [65] and NWPU-RESISC45 [66].”
Uformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
Luo, W., Li, Y., Urtasun, R., & Zemel, R. (2017) · arXiv · doi:10.48550/arxiv.1701.04128 · Cited by 1,575
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs
Ding, X., Zhang, X., Han, J., & Ding, G. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01166 · Cited by 1,447
Anchored Neighborhood Regression for Fast Example-Based Super-Resolution
Timofte, R., De, V., & Gool, L. V. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.241 · Cited by 1,351
Deep Learning for Single Image Super-Resolution: A Brief Review
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.-H., & Liao, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2919431 · Cited by 1,154
Activating More Pixels in Image Super-Resolution Transformer
Chen, X., Wang, X., Zhou, J., Qiao, Y., & Dong, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02142 · Cited by 1,152
Guo, M.-H., Lu, C.-Z., Liu, Z.-N., Cheng, M.-M., & Hu, S.-M. (2023) · Computational Visual Media · doi:10.1007/s41095-023-0364-2 · Cited by 1,073
Learning Texture Transformer Network for Image Super-Resolution
Yang, F., Yang, H., Fu, J., Lu, H., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00583 · Cited by 966
Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior
Kim, K. I., & Kwon, Y. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.25 · Cited by 956
Single Image Super-Resolution via a Holistic Attention Network
Niu, B., Wen, W., Ren, W., Zhang, X., Yang, L., Wang, S., Zhang, K., Cao, X., & Shen, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_12 · Cited by 748
Transformer for Single Image Super-Resolution
Lu, Z., Li, J., Liu, H., Huang, C., Zhang, L., & Zeng, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00061 · Cited by 678
Self-citationImage Super-Resolution with Non-Local Sparse Attention
Mei, Y., Fan, Y., & Zhou, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00352 · Cited by 607
Edge-Enhanced GAN for Remote Sensing Image Superresolution
Jiang, K., Wang, Z., Yi, P., Wang, G., Lu, T., & Jiang, J. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2902431 · Cited by 545
Self-citationDual Aggregation Transformer for Image Super-Resolution
Chen, Z., Zhang, Y., Gu, J., Kong, L., Yang, X., & Yu, F. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01131 · Cited by 459
Xiao, J., Fu, X., Liu, A., Wu, F., & Zha, Z.-J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183612 · Cited by 389
Super-Resolution for Remote Sensing Images via Local–Global Combined Network
Lei, S., Shi, Z., & Zou, Z. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2704122 · Cited by 377
SRFormer: Permuted Self-Attention for Single Image Super-Resolution
Zhou, Y., Li, Z., Guo, C.-L., Bai, S., Cheng, M.-M., & Hou, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01174 · Cited by 374
Interpreting Super-Resolution Networks with Local Attribution Maps
Gu, J., & Dong, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00908 · Cited by 361
Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement
Sun, J., Sun, J., Xu, Z., & Shum, H.-Y. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2095871 · Cited by 341
EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution
Xiao, Y., Yuan, Q., Jiang, K., He, J., Jin, X., & Zhang, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3341437 · Cited by 280
Self-citationHierarchical dense recursive network for image super-resolution
Jiang, K., Wang, Z., Yi, P., & Jiang, J. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107475 · Cited by 243
Self-citationTransformer-Based Multistage Enhancement for Remote Sensing Image Super-Resolution
Lei, S., Shi, Z., & Mo, W. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3136190 · Cited by 236
Xiao, Y., Yuan, Q., Jiang, K., He, J., Wang, Y., & Zhang, L. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.021 · Cited by 229
Self-citationA Hybrid Network of CNN and Transformer for Lightweight Image Super-Resolution
Fang, J., Lin, H., Chen, X., & Zeng, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00119 · Cited by 207
KVT: k-NN Attention for Boosting Vision Transformers
Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., & Jin, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_17 · Cited by 198
NFormer: Robust Person Re-identification with Neighbor Transformer
Wang, H., Shen, J., Liu, Y., Gao, Y., & Gavves, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00715 · Cited by 194
Hyperspectral Image Denoising: From Model-Driven, Data-Driven, to Model-Data-Driven
Zhang, Q., Zheng, Y., Yuan, Q., Song, M., Yu, H., & Xiao, Y. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3278866 · Cited by 189
Self-citationSwin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
Conde, M. V., Choi, U.-J., Burchi, M., & Timofte, R. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25063-7_42 · Cited by 180
Local-Global Temporal Difference Learning for Satellite Video Super-Resolution
Xiao, Y., Yuan, Q., Jiang, K., Jin, X., He, J., Zhang, L., & Lin, C.-W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3312321 · Cited by 180
Self-citationHybrid-Scale Self-Similarity Exploitation for Remote Sensing Image Super-Resolution
Lei, S., & Shi, Z. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3069889 · Cited by 173
Cooperated Spectral Low-Rankness Prior and Deep Spatial Prior for HSI Unsupervised Denoising
Zhang, Q., Yuan, Q., Song, M., Yu, H., & Zhang, L. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3211471 · Cited by 167
Self-citationRan, R., Deng, L.-J., Jiang, T.-X., Hu, J.-F., Chanussot, J., & Vivone, G. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3238200 · Cited by 167
Xiao, Y., Su, X., Yuan, Q., Liu, D., Shen, H., & Zhang, L. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3107352 · Cited by 160
Self-citationLin, M., Yang, G., & Zhang, H. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3226418 · Cited by 159
Deep Distillation Recursive Network for Remote Sensing Imagery Super-Resolution
Jiang, K., Wang, Z., Yi, P., Jiang, J., Xiao, J., & Yao, Y. (2018) · Remote Sensing · doi:10.3390/rs10111700 · Cited by 138
Self-citationLRTCFPan: Low-Rank Tensor Completion Based Framework for Pansharpening
Wu, Z.-C., Huang, T., Deng, L.-J., Huang, J. W., Chanussot, J., & Vivone, G. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3247165 · Cited by 126
Spectral super-resolution meets deep learning: Achievements and challenges
He, J., Yuan, Q., Li, J., Xiao, Y., Liu, D., Shen, H., & Zhang, L. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101812 · Cited by 126
Self-citationXiao, Y., Yuan, Q., He, J., Zhang, Q., Sun, J., Su, X. B., Wu, J., & Zhang, L. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102731 · Cited by 120
Self-citationDeep Bilateral Filtering Network for Point-Supervised Semantic Segmentation in Remote Sensing Images
Wu, L., Fang, L., Yue, J., Zhang, B., Ghamisi, P., & He, M. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3222904 · Cited by 118
Single-frame super-resolution in remote sensing: a practical overview
Fernández-Beltrán, R., Carmona, P., & Pla, F. (2016) · International Journal of Remote Sensing · doi:10.1080/01431161.2016.1264027 · Cited by 117
SAGN: Semantic-Aware Graph Network for Remote Sensing Scene Classification
Yang, Y., Tang, X., Cheung, Y., Zhang, X., & Jiao, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3238310 · Cited by 93
HIPA: Hierarchical Patch Transformer for Single Image Super Resolution
Cai, Q., Qian, Y., Li, J., Lyu, J., Yang, Y., Wu, F., & Zhang, D. D. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3279977 · Cited by 80
Hybrid Attention-Based U-Shaped Network for Remote Sensing Image Super-Resolution
Wang, J., Wang, B., Wang, X., Zhao, Y., & Long, T. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3283769 · Cited by 74
DsTer: A dense spectral transformer for remote sensing spectral super-resolution
He, J., Yuan, Q., Li, J., Xiao, Y., Liu, X., & Zou, Y. L. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102773 · Cited by 70
Self-citationInterpretable Detail-Fidelity Attention Network for Single Image Super-Resolution
Huang, Y., Li, J., Gao, X., Hu, Y., & Lu, W. B. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3050856 · Cited by 69
Deep Blind Super-Resolution for Satellite Video
Xiao, Y., Yuan, Q., Zhang, Q., & Zhang, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3291822 · Cited by 65
Self-citationYang, J., Du, B., Xu, Y., et al. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3244414 · Cited by 62
Meta-Learning-Based Degradation Representation for Blind Super-Resolution
Xia, B., Tian, Y., Zhang, Y., Hang, Y., Yang, W., & Liao, Q. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3283922 · Cited by 44
Super-Resolution Phase Retrieval Network for Single-Pattern Structured Light 3D Imaging
Song, J., Liu, K., Sowmya, A., et al. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3230245 · Cited by 39
AST: Adaptive Self-supervised Transformer for optical remote sensing representation
He, Q., Sun, X., Yan, Z., Wang, B., Zhu, Z., Diao, W., & Yang, M. Y. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.04.003 · Cited by 38
Attention Retractable Frequency Fusion Transformer for Image Super Resolution
Zhu, Q., Li, P., & Li, Q. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00176 · Cited by 36
Progressive Context-Dependent Inference for Object Detection in Remote Sensing Imagery
Liu, B., Xu, C., Cui, Z., & Yang, J. J. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3231058 · Cited by 25
Task-Agnostic Vision Transformer for Distributed Learning of Image Processing
Kim, B., Kim, J., & Ye, J. C. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3226892 · Cited by 22
Taylor Neural Network for Real-World Image Super-Resolution
Wei, P., Xie, Z., Li, G., & Lin, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3255107 · Cited by 22
Transitional Learning: Exploring the Transition States of Degradation for Blind Super-Resolution
Huang, Y., Li, J., Hu, Y., Gao, X., & Huang, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206870 · Cited by 12
Metadata
- Authors
- Xiao, Y., Yuan, Q., Jiang, K., He, J., Lin, C.-W., & Zhang, L.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2024
- Volume
- 33
- Pages
- 738-752
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W4390706297
- Cited by
- 413
- References
- 70
- FWCI
- 58.34
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.