The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 201–300 of 1,001
Liu, P., Qian, W., Zhang, H., Zhu, Y., Hong, Q., Li, Q., & Yao, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10926-9 · Cited by 48
Deng, P., Han, X., Wei, X., & Chang, L. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106172 · Cited by 36
LKAU-Net: 3D Large-Kernel Attention-Based U-Net for Automatic MRI Brain Tumor Segmentation
Li, H., Yang, N., & Yang, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-12053-4_24 · Cited by 35
Transformer-Based Dual-Channel Self-Attention for UUV Autonomous Collision Avoidance
Lin, C., Cheng, Y., Wang, X., Yuan, J., & Wang, G. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3245615 · Cited by 35
WiTransformer: A Novel Robust Gesture Recognition Sensing Model with WiFi
Yang, M., Zhu, H., Zhu, R., Wu, F., Yin, L., & Yang, Y. (2023) · Sensors · doi:10.3390/s23052612 · Cited by 40
“In fact, attention has been evolving for over 10 years [51].”
BackgroundGe, W., Yang, X., Jiang, R., Shao, W., & Zhang, L. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3361933 · Cited by 38
“…complex scenes, attention mechanism, a dynamic weight adjustment process based on feature maps, was introduced into the field of CV. Attention mechanisms can be divided into many types according to approach [37], such as channel attention, spatial attention, temporal attention, and branch attention.”
BackgroundMishra, L. N., & Senapati, B. (2025) · IEEE Access · doi:10.1109/access.2025.3552592 · Cited by 35
“In the conceptual development of the transformer model, the relationship among token t i , mechanism computes query ( q i ), key ( k i ), and value ( v i ) vectors are defined by Equation 8 [45].”
Zhou, G., Zhi, H., Gao, E., Lu, Y., Chen, J., Bai, Y., & Zhou, X. H. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3555636 · Cited by 38
“U-Net excel-lently performs in handling small targets [24], [25], particularly under high-resolution images, through detailed recovery [26].”
Auguring Fake Face Images Using Dual Input Convolution Neural Network
Bhandari, M., Neupane, A., Mallik, S., Gaur, L., & Qin, H. H. (2022) · Journal of Imaging · doi:10.3390/jimaging9010003 · Cited by 36
“The incredible advancements that have been made in deep learning (DL) research have made it possible to resolve complex tasks in computer vision [3], including neural network optimization, natural language processing [4], image processing [5], intelligent transportation [6], and image steganography [7].”
BackgroundAhmed, M. R., Ashrafi, A. F., Ahmed, R. U., Shatabda, S., Islam, A. K. M. M., & Islam, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08493-1 · Cited by 38
Crowd counting in smart city via lightweight Ghost Attention Pyramid Network
Guo, X., Song, K., Gao, M., Zhai, W., Li, Q., & Jeon, G. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.05.013 · Cited by 34
Park, M., Tran, D. Q., Bak, J., Kulinan, A. S., & Park, S. (2023) · Journal of Safety Research · doi:10.1016/j.jsr.2023.08.018 · Cited by 37
Unsupervised Domain Adaptation for Medical Image Segmentation Using Transformer With Meta Attention
Ji, W., & Chung, A. C. S. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3322581 · Cited by 35
“After these applications to NLP, the attention mechanism also demonstrated its power in computer vision [34], such as image caption [35], image classification [14], semantic segmentation [16], image generation [36], etc.”
MethodImproved object detection via large kernel attention
Wang, Z., Li, Y., Liu, Y., & Meng, F. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122507 · Cited by 35
Yang, R., Fu, Y., Zhang, Q., & Zhang, L. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102805 · Cited by 34
Efficient image denoising with heterogeneous kernel-based CNN
Hu, Y., Tian, C., Zhang, J., & Zhang, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127799 · Cited by 34
Self-citationPark, M., Kulinan, A. S., Tran, D. Q., Bak, J., & Park, S. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105536 · Cited by 35
Jácome-Galarza, L.-R., Realpe, M., Viñán-Ludeña, M. S., Calderón, F., & Luzuriaga, S. A. J. (2025) · Electronics · doi:10.3390/electronics14122466 · Cited by 34
Li, Q., Wang, Z., Wang, M., Zhao, J., Tu, K., Lan, W., Liu, J., & Pan, L. (2025) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.70248 · Cited by 34
Self-citationCross-domain heterogeneous residual network for single image super-resolution
Li, J., Zhu, Q., Zhang, Y., Yin, J., Wei, R., Xiao, J., Xiao, D., & Zhao, G. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.02.008 · Cited by 34
Workshop Safety Helmet Wearing Detection Model Based on SCM-YOLO
Zhang, B., Sun, C.-F., Fang, S., Zhao, Y.-H., & Su, S. (2022) · Sensors · doi:10.3390/s22176702 · Cited by 36
“( ) F s M is shown in Formula (7): [ ] where σ represents the sigmoid function, and”
Xiang, J., Wang, Y., An, L., Liu, H., Wang, Z., & Liu, J. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3210298 · Cited by 38
“Attention mechanism is a dynamic process that diverts attention to important features [36].”
BackgroundSelf-citationImproved YOLOv5 for real-time traffic signs recognition in bad weather conditions
Phuc, D. T., Tran, N. T., To, V. H., & Thi, M. K. T. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05097-3 · Cited by 39
Chen, Z., Yang, D., Cui, T., Li, D., Liu, H., Yang, Y., Sheng, Z., Yang, S., & Ren, T. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105437 · Cited by 41
Diffusion-Based Visual Art Creation: A Survey and New Perspectives
Wang, B., Chen, Q., & Wang, Z. (2025) · ACM Computing Surveys · doi:10.1145/3728459 · Cited by 40
Yu, Z., Ye, J., Li, C., Zhou, H., & Li, X. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1158940 · Cited by 36
“They tend to be less computationally expensive than CNNs due to their lower structural complexity and number of parameters.”
BackgroundInfluentialGuo, Y., Peng, Y., Hao, R., & Tang, X. (2023) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2023.103746 · Cited by 33
Mangrove extraction from super-resolution images generated by deep learning models
Hong, Y., Que, X., Wang, Z., Ma, X., Wang, H., Salati, S., & Liu, J. (2024) · Ecological Indicators · doi:10.1016/j.ecolind.2024.111714 · Cited by 32
Self-citationZhou, Z., Le, Y., Wang, Y., Tian, Y., & Li, X. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142605 · Cited by 33
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 58
“In addition, a considerable amount of fusion technologies has focused on the attention mechanism [303, 186].”
BackgroundSelf-citationTheoretically Achieving Continuous Representation of Oriented Bounding Boxes
Xiao, Z.-K., Yang, G.-Y., Yang, X., Mu, T., Yan, J., & Hu, S.-M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01600 · Cited by 36
“With the increasing adoption of deep learning in computer vision, object detection models [5, 6, 11, 16, 24, 30, 53] have emerged to enhance computers’ capacity for recognizing objects in natural images.”
BackgroundSelf-citationA Review of Intelligent Orchard Sprayer Technologies: Perception, Control, and System Integration
Wu, M., Liu, S., Li, Z., Ou, M., Dai, S., Dong, X. B., Wang, X., Jiang, L. Y., & Jia, W. (2025) · Horticulturae · doi:10.3390/horticulturae11060668 · Cited by 33
Machine learning in cartography
Harrie, L., Touya, G., Oucheikh, R., Ai, T., Courtial, A., & Richter, K. (2024) · Cartography and Geographic Information Science · doi:10.1080/15230406.2023.2295948 · Cited by 39
“Two recent review papers give a good idea of these flour-ishing new applications (Guo et al., 2022; Khan et al., 2022) with possible implications for cartography.”
BackgroundA novel feature attention mechanism for improving the accuracy and robustness of runoff forecasting
Wang, H. E., Qin, H., Liu, G., Liu, S., Qu, Y., Wang, K., & Zhou, J. (2023) · Journal of Hydrology · doi:10.1016/j.jhydrol.2023.129200 · Cited by 34
Transformer network for data imputation in electricity demand data
Lotfipoor, A., Patidar, S., & Jenkins, D. P. (2023) · Energy and Buildings · doi:10.1016/j.enbuild.2023.113675 · Cited by 33
Alrfou, K., Zhao, T., & Kordijazi, A. (2024) · Integrating materials and manufacturing innovation · doi:10.1007/s40192-024-00369-z · Cited by 49
“However, the local receptive field of a CNN inherently limits its ability to model long-range dependencies within an image [11, 36].”
Guo, Y., Yi-Xin, Z., Gao, F., Qiu, Y., Zhou, X., Zhong, L., & Zhan, C. (2022) · IEEE Access · doi:10.1109/access.2022.3152552 · Cited by 37
“Squeeze-and-excitation networks (SENet) [38] is the pioneer of channel attention, which generates attention mask across the channel domain and use it to select essential channels [39].”
BackgroundMulti-Label Fundus Image Classification Using Attention Mechanisms and Feature Fusion
Li, Z., Xu, M., Yang, X., & Han, Y. (2022) · Micromachines · doi:10.3390/mi13060947 · Cited by 32
“This module imitates humans in finding ignificant regions in complicated situations a d has applications in a variety of vision tasks [21], including image classification, target identification, image segmentation, and facial recognition.”
BackgroundAn Adaptive ORB-SLAM3 System for Outdoor Dynamic Environments
Zang, Q., Zhang, K., Wang, L., & Wu, L. (2023) · Sensors · doi:10.3390/s23031359 · Cited by 33
“Adjusting the dynamic feature weights of the input image is called the attention mechanism [31].”
BackgroundYang, Z., Wu, Q., Zhang, F., Zhang, X., Chen, X.-F., & Gao, Y. (2023) · Symmetry · doi:10.3390/sym15051037 · Cited by 35
“…problem of inadequate learning VGG16 ISPRS Vaihingen 74.63 Moreover, attention mechanisms have been successfully applied in semantic segmentation [30,31] over the past few years, where introducing an attention mechanism into a semantic segmentation model allows the model to better focus on…”
InfluentialThakuria, A., & Erkinbaev, C. (2023) · Smart Agricultural Technology · doi:10.1016/j.atech.2023.100300 · Cited by 32
Liang, Y., Ren, Y., Yu, J., & Zha, W. (2023) · Protection and Control of Modern Power Systems · doi:10.1186/s41601-023-00280-3 · Cited by 31
“The fundamental thought behind the attention mechanism is that the model learns to concentrate on important information and ignore inconse-quential information [23, 24].”
BackgroundHu, Y.-C., Tian, S., & Ge, J. (2023) · Remote Sensing · doi:10.3390/rs15194796 · Cited by 31
Kim, Y., & Kim, Y.-K. (2023) · Sensors · doi:10.3390/s23239311 · Cited by 42
“Recently, the attention concept has been applied to image processing with CNN models to focus more on highly meaningful features [45].”
MethodCTH-Net: A CNN and Transformer hybrid network for skin lesion segmentation
Ding, Y., Yi, Z., Xiao, J., Hu, M., Guo, Y. F., Liao, Z., & Wang, Y. (2024) · iScience · doi:10.1016/j.isci.2024.109442 · Cited by 35
Zhang, X., Xiao, Z., Ma, J., Wu, X., Zhao, J., Zhang, S., Li, R., Pan, Y. F., & Liu, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3551128 · Cited by 32
“Researchers have demonstrated the e ff ectiveness of attention mechanisms in a wide range of learning tasks such as image classification and image segmentation [27], [28].”
Self-citationPham, T.-N., Nguyen, V.-H., & Huh, J. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-022-04979-2 · Cited by 31
“Besides, adding attention mechanisms into deep learning models for image classification has proven to be helpful [23].”
BackgroundJin, Y., Gao, F., Yu, J., Wang, J., & Shuang, F. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3249162 · Cited by 32
Dodda, V. C., Kuruguntla, L., Mandpura, A. K., & Elumalai, K. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3267037 · Cited by 31
“The purpose of attention module is to adaptively recalibrate the weight of each channel and improve the representation ability [59].”
BackgroundWei, Y., Zhu, Y., Zhou, Y., Yu, X., & Luo, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3332503 · Cited by 31
“Furthermore, a number of lightweight and universal attention modules provide more possibilities for an efficient network in automatic sleep staging [20],whichcouldassignhigherweightstotheregionsofinterest.”
BackgroundElectromyographic hand gesture recognition using convolutional neural network with multi-attention
Zhang, Z., Shen, Q., & Wang, Y. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105935 · Cited by 33
Zhao, K., Lu, R., Wang, S., Yang, X., Li, Q., & Fan, J. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1170163 · Cited by 37
“Usually, attention mechanisms in the vision domain (Guo M. et al., 2022) include two types: spatial and channel.”
BackgroundDeep Representation Learning: Fundamentals, Technologies, Applications, and Open Challenges
Payandeh, A., Baghaei, K. T., Fayyazsanavi, P., Ramezani, S. B., Chen, Z., & Rahimi, S. (2023) · IEEE Access · doi:10.1109/access.2023.3335196 · Cited by 49
“By dynamically assigning different weights to different parts of the input sequence, attention mechanisms allow the models to capture dependencies and relationships effectively [45], [336].”
BackgroundAghapanah, H., Rasti, R., Kermani, S., Tabesh, F., Yousefi-Banaem, H., Aliakbar, H. P., Sanei, H., & Segars, W. P. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102382 · Cited by 30
Yang, X. Q., & Mohamed, A. S. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129248 · Cited by 34
Uncertainty quantification and attention-aware fusion guided multi-modal MR brain tumor segmentation
Zhou, T., & Zhu, S. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107142 · Cited by 33
Efficient deep learning models based on tension techniques for sign language recognition
Attia, N. F., Ahmed, M. T. F. S., & Alshewimy, M. A. M. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200284 · Cited by 36
Lightweight convolutional neural network for counting densely piled steel bars
Liu, H., Wang, D., Xu, K., Zhou, P., & Zhou, D. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104692 · Cited by 29
Zarbakhsh, P. (2023) · Applied Sciences · doi:10.3390/app13158758 · Cited by 29
Motion-Oriented Hybrid Spiking Neural Networks for Event-Based Motion Deblurring
Liu, Z., Wu, J., Shi, G., Yang, W. C., Dong, W., & Zhao, Q. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3317976 · Cited by 32
“Its significance has been studied extensively in the previous literature [46], [47], [48], [49].”
BackgroundSelf-citationLee, J. H., & You, S. J. (2024) · IEEE Access · doi:10.1109/access.2024.3352146 · Cited by 29
“RT-DETR relies on its inherent self-attention mechanism for learning object relationships, which may pose challenges when certain classes are anonymized in the training data, rendering objects less distinguishable [70].”
BackgroundSelf-attention and long-range relationship capture network for underwater object detection
Gao, Z., Shi, Y., & Li, S. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.101971 · Cited by 31
“In deep neural networks, different channels often represent different objects (Chen, 2017; Guo, 2022), and adaptation in the channel dimension is equally essential for vision tasks.”
Li, X., Zhang, Z., Li, Q., & Zhu, J. (2024) · Water · doi:10.3390/w16101376 · Cited by 29
Attention activation network for bearing fault diagnosis under various noise environments
Zhang, Y., Lin, L., Wang, J., Zhāng, W., Gao, S., & Zhang, Z. (2025) · Scientific Reports · doi:10.1038/s41598-025-85275-w · Cited by 30
Lin, H., Tse, R. T., Tang, S.-K., Qiang, Z., & Pau, G. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.907916 · Cited by 32
“Attention mechanism plays important role in human perception to selectively focus on salient parts in order to capture visual structure better (Guo et al., 2021).”
BackgroundADS-detector: An attention-based dual stream adversarial example detection method
Guo, S., Li, X., Zhu, P., & Mu, Z. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110388 · Cited by 29
BEVSOC: Self-Supervised Contrastive Learning for Calibration-Free BEV 3-D Object Detection
Chen, Y., Li, N., Zhu, D., Zhou, C., Hu, Z., Bai, Y., & Yan, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3379471 · Cited by 29
“DETR3D [43] referred to DETR [3] used attention mechanism [12] for 3-”
MethodFan, Y., Qian, Y., Qin, Y., Wan, Y., Gong, W., Zhuang, C., & Liu, H. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3221098 · Cited by 31
“…attention mechanisms have been introduced into computer vision systems, and more and more deep learning models based on attention mechanisms have been proposed and improved feature representation in many research areas (e.g., image classification, object detection, image generation, etc.) [33].”
BackgroundData Augmentation and Few-Shot Change Detection in Forest Remote Sensing
Zhu, S., Jing, W., Kang, P., Emam, M., & Li, C. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3285389 · Cited by 28
Zhang, Q., He, C., He, B., & Tong, M. H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3273392 · Cited by 28
“Attention mechanisms can be categorized based on the data domain into four basic categories (channel attention, spatial attention, temporal attention, and branch attention) and two hybrid categories (channel and spatial attention, and spatial and temporal attention) [48].”
BackgroundTwo-stream cross-attention vision Transformer based on RGB-D images for pig weight estimation
He, W. P., Mi, Y., Ding, X., Liu, G., & Li, T. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.107986 · Cited by 28
AGL-Net: An Efficient Neural Network for EEG-Based Driver Fatigue Detection
Fang, W., Tang, L., & Pan, J. (2023) · Journal of Integrative Neuroscience · doi:10.31083/j.jin2206146 · Cited by 28
“The attention module adaptively learns key information from the input sequences to improve the network’s sensitivity and prediction performance [37].”
Xiao, Z., Zhang, X., Zheng, B., Guo, Y., Higashita, R., & Liu, J. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107958 · Cited by 29
Self-citationKalopesa, E., Gkrimpizis, T., Samarinas, N., Tsakiridis, N., & Zalidis, G. C. (2023) · Sensors · doi:10.3390/s23239536 · Cited by 29
“In the realm of artificial intelligence (AI), attention layers [18], a fundamental component in the realm of deep learning, have witnessed significant adoption and innovation in recent years [19], particularly in the context of natural language processing and computer vision tasks [20].”
BackgroundKianNet: A Violence Detection Model Using an Attention-Based CNN-LSTM Structure
Vosta, S., & Yow, K. (2023) · IEEE Access · doi:10.1109/access.2023.3339379 · Cited by 30
“Primarily, this can dynamically adjust the weight of input image features [10].”
BackgroundApplications of interpretable deep learning in neuroimaging: A comprehensive review
Munroe, L., da Silva, M., Heidari, F., Grigorescu, I., Dahan, S., Robinson, E. C., Deprez, M., & So, P. (2024) · Imaging Neuroscience · doi:10.1162/imag_a_00214 · Cited by 33
“For a more comprehensive description of DL attention mechanisms, we refer the reader to Niu et al. (2021) and Guo et al. (2022).”
BackgroundGradient-Guided Multiscale Focal Attention Network for Remote Sensing Scene Classification
Zhao, Y., Gong, M., Qin, A. K., Zhang, M., Hu, Z., Gao, T., & Pu, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3424489 · Cited by 32
“The attention mechanism is inspired by the concept in cognitive science that humans selectively focus on a portion of all information while ignoring other visible information due to a bottleneck in information processing, which has been extensively practiced in computer vision [48].”
BackgroundSun, Z., Zhang, J., Chen, Z., Lu, H., Zhang, R., Li, W., & Xia, H. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108859 · Cited by 29
Longyang, Q., Choi, S., Tennant, H., Hill, D. S., Ashmead, N., Neilson, B. T., Newell, D. L., McNamara, J. P., & Xu, T. (2024) · Water Resources Research · doi:10.1029/2024wr037878 · Cited by 30
“To emulate the selective focus seen in human perception, spatial attention seeks to dynamically adjust the weights assigned to image features output by preceding layers (Guo et al., 2022).”
Self-citationLi, G., Zhao, S., Zhou, M., Li, M., Shao, R., Zhang, Z., & Han, D. (2022) · Electronics · doi:10.3390/electronics11244211 · Cited by 28
Semantic segmentation for remote sensing images based on an AD-HRNet model
Yang, X., Fan, X., Peng, M., Guan, Q., & Tang, L. (2022) · International Journal of Digital Earth · doi:10.1080/17538947.2022.2159080 · Cited by 28
“The channel attention mechanism evaluates the importance of each feature channel and then enhances or suppresses di ff erent channels for di ff erent categories to determine the attention position (Guo et al. 2022).”
BackgroundA Multi-Attention Autoencoder for Hyperspectral Unmixing Based on the Extended Linear Mixing Model
Su, L., Liu, J., Yuan, Y., & Chen, Q. (2023) · Remote Sensing · doi:10.3390/rs15112898 · Cited by 28
Self-citationYao, M., Zhang, H., Zhao, G., Zhang, X., Wang, D., Cao, G., & Li, G. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.07.008 · Cited by 30
DeepRSSI: Generative Model for Fingerprint-Based Localization
Yoon, N., Jung, W., & Kim, H. (2024) · IEEE Access · doi:10.1109/access.2024.3398734 · Cited by 27
“The attention mechanism is a technique for efficient calculation by selectively focusing on specific features or areas of input data [31].”
MethodWang, Z., Zhu, S., Li, J., Jiang, W., Ramakrishnan, K. K., Yan, M. L., Zhang, X., & Liu, A. X. (2024) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2024.3400953 · Cited by 27
“To avoid overfitting, a normalization layer [42] is used after each convolutional block.”
MethodA lightweight YOLOv8 based on attention mechanism for mango pest and disease detection
Wang, J., & Wang, J. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01505-w · Cited by 27
“With the recent rise of computer vision, attention mechanisms have been widely applied and achieved great success [29].”
Zhu, D., Han, Z., Du, X. H., Zuo, D., Cai, L., & Xue, C. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3472043 · Cited by 27
Hypergraph BiFormer for Semantic Segmentation of High-Resolution Remote Sensing Images
Jing, W., Zhang, W., Di, D., Li, C., Emam, M., & Mian, A. S. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3543556 · Cited by 29
“This can be succinctly categorized into what to allocate attention to (channel attention), where to direct attention (spatial attention), when to focus attention (temporal attention), and the amalgamation of these aspects [42].”
Spatial–Spectral Split Attention Residual Network for Hyperspectral Image Classification
Shu, Z., Liu, Z., Zhou, J. W., Tang, S., Yu, Z., & Wu, X. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3225928 · Cited by 27
“Motivated by the human visual theory, the attention mechanism was proposed to selectively learn significant targets in image sample [32], [33].”
BackgroundSelf-citationD2ANet: Difference-aware attention network for multi-level change detection from satellite imagery
Mei, J., Zheng, Y., & Cheng, M. (2023) · Computational Visual Media · doi:10.1007/s41095-022-0325-1 · Cited by 32
“Guo et al. [68] provided a systematic review of visual attention methods.”
MethodSelf-citationXiao, H., Liu, Q., & Li, L. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104768 · Cited by 28
A lightweight neural network for lung nodule detection based on improved ghost module
Yang, L., Cai, H., Luo, X., Wu, J., Tang, R., Chen, Y., & Li, W. (2023) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-1182 · Cited by 27
YOLOv7-CHS: An Emerging Model for Underwater Object Detection
Zhao, L., Yun, Q., Yuan, F., Ren, X., Jin, J., & Zhu, X. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11101949 · Cited by 27
“To demonstrate the effectiveness and versatility of the proposed in this paper, we chose two underwater imaging datasets, namely, the Starfish dataset and the DUO dataset, for validation purposes. the model’s performance in different underwater scenarios.”
MethodInfluentialDual Spatial Attention Network for Underwater Object Detection With Sonar Imagery
Li, Z., Xie, Z., Duan, P., Kang, X., & Shutao Li, A. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3336899 · Cited by 28
“Therefore, there are many studies on attention mechanisms [40], [41], [44] in recent years, which attention modules make the network focus on areas that need to be noticed and have achieved good performance in the conventional object detection task.”
BackgroundA Spectral–Spatial Context-Boosted Network for Semantic Segmentation of Remote Sensing Images
Li, X., Yong, X., Li, T., Tong, Y., Gao, H., Wang, X., Xu, Z., Fang, Y., You, Q., & Lyu, X. (2024) · Remote Sensing · doi:10.3390/rs16071214 · Cited by 27
Intelligent Video Analytics for Human Action Recognition: The State of Knowledge
Kulbacki, M., Segen, J., Chaczko, Z., Rozenblit, J. W., Kulbacki, M., Klempous, R., & Wojciechowski, K. W. (2023) · Sensors · doi:10.3390/s23094258 · Cited by 31
Boundary-Aware Multiscale Learning Perception for Remote Sensing Image Segmentation
You, C., Jiao, L., Liu, X., Li, L., Liu, F., Ma, W., & Yang, S. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3300706 · Cited by 26
“Inspired by human perception, the attention mechanisms can guide networks to find salient regions of scenes [47], which can greatly improve the representation of features.”
BackgroundWang, M., Wang, H., & Zhang, F. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614876 · Cited by 26
YOLO-PAM: Parasite-Attention-Based Model for Efficient Malaria Detection
Zedda, L., Loddo, A., & Ruberto, C. D. (2023) · Journal of Imaging · doi:10.3390/jimaging9120266 · Cited by 26
“With regard to CV tasks, attention mechanisms can be broadly classified into channel, spatial, temporal, branch, channel and spatial, and spatial and temporal attention [61].”
BackgroundInfluentialMotion Deblur by Learning Residual From Events
Chen, K., & Yu, L. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3355630 · Cited by 30
“Attention was initially proposed in natural language processing [36], [37] and recently achieved superior performance in various computer vision tasks [38], [39].”
Background
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
