The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 701–800 of 1,001
RDASNet: Image Denoising via a Residual Dense Attention Similarity Network
Tao, H., Guo, W., Han, R., Yang, Q. Z., & Zhao, J. (2023) · Sensors · doi:10.3390/s23031486 · Cited by 12
“The attention mechanism used in this study originated in the human brain [23], then it was introduced into natural language processing [24] and applied to computer vision [25].”
MethodA survey on attention mechanisms for medical applications: are we moving towards better algorithms?Preprint
Gonçalves, T., Rio-Torto, I., Teixeira, L. F., & Cardoso, J. S. (2022) · Research Square · doi:10.21203/rs.3.rs-1594205/v1 · Cited by 11
Advancements in High-Resolution Computed Tomography: Revolutionising Bone Health Micro-Research
Lindtner, R. A., Kampik, L., Putzer, D., Klosterhuber, M., Pallua, A., Streif, W., Schirmer, M., Degenhart, G., Arora, R. R., & Pallua, J. D. (2025) · Bioengineering · doi:10.3390/bioengineering12111189 · Cited by 13
Xu, B., Li, H., Liu, Y., Zhou, F., & Baokang, Y. (2023) · Measurement Science and Technology · doi:10.1088/1361-6501/acf681 · Cited by 10
“Transformer network [44, 45] replaces cyclic CNN with the structure of full attention mechanism, which better solves the series-to-sequence machine translation problem and has been applied in fault diagnosis [46].”
Zhao, J., Tan, J., Chen, S., Huang, Q., Gao, L., Li, Y., & Wei, C. (2024) · Remote Sensing · doi:10.3390/rs16020275 · Cited by 10
An Improved Nested U-Net Network for Fluorescence In Situ Hybridization Cell Image Segmentation
Jian, Z., Song, T., Zhang, Z., Ai, Z., Zhao, H., Tang, M., & Liu, K. (2024) · Sensors · doi:10.3390/s24030928 · Cited by 10
“This method enables humans to efficiently and accurately select and focus on valuable information amidst a large amount of data [17,18].”
BackgroundOttakath, N., Akbari, Y., Al-Maadeed, S. A., Bouridane, A., Zughaier, S. M., & Chowdhury, M. E. H. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106350 · Cited by 10
NLKFill: high-resolution image inpainting with a novel large kernel attention
Wang, T., Xiang, D., Yang, C., Liang, J., & Shi, C. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01411-5 · Cited by 10
Image Classification Using Hybrid Classical-Quantum Neutral Networks
Ling, Y.-Q., Zhang, J., Zhang, L., Li, Y.-R., & Huang, H.-L. (2024) · International Journal of Theoretical Physics · doi:10.1007/s10773-024-05669-w · Cited by 13
“With the rapid development of information technology, image classification holds a central position in the field of computer vision, and plays a crucial role in many application scenarios [1].”
Hierarchical Encoding Method for Retinal Segmentation Evolutionary Architecture Search
Sheng, H., Liu, H., Lai, Y., Zeng, S., & Chen, L. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3395540 · Cited by 10
“In visual tasks, different channels often correspond to distinct objects or features [35], [36].”
Zhang, G., Li, Q., Gao, M., Guo, S., & Jeon, G. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3414319 · Cited by 10
“Many attention mechanisms are proposed in the literature [21].”
EEMNet: an end-to-end efficient model for PCB surface tiny defect detection
Wu, Y., Zheng, L., & Chen, E. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02264-7 · Cited by 10
Ding, J.-T., Peng, Y.-Y., Huang, M., & Zhou, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-65540-0 · Cited by 10
Multi-scale image compression and reconstruction algorithm for structural health monitoring system
Shen, W., Tian, X., Zeng, D., & Zhang, Y. (2024) · Engineering Structures · doi:10.1016/j.engstruct.2024.118516 · Cited by 11
FP-Deeplab: a segmentation model for fabric defect detection
Liu, Y., Shen, J., Ye, R., Wang, S., Ren, J., & Pan, H. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad5f50 · Cited by 11
Li, X., Su, Y., Yao, J., Gu, H., & Jiao, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112215 · Cited by 10
Hu, Z., Chen, H., Long, H., Ren, X., & Mei, W. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106861 · Cited by 10
Frequency Regulated Channel-Spatial Attention module for improved image classification
Zhuang, C., Yuan, X., Gu, L., Wei, Z., Fan, Y., & Guo, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125463 · Cited by 11
Image-Based Fitness Yoga Pose Recognition: Using Ensemble Learning and Multi-head Attention
Kou, Y., & Li, H. H. (2024) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-024-00662-x · Cited by 10
“Research on yoga pose recognition technology focuses on traditional machine learning-based yoga pose recognition, deep learning-based yoga pose image classifi-cation, and deep learning-based yoga pose detection [11].”
Self-Supervised Marine Organism Detection From Underwater Images
Li, J., Yang, W., Qiao, S., Gu, Z., Zheng, B., & Zheng, H. (2024) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2024.3455565 · Cited by 12
A systematic review of deep learning for structural geological interpretation
Fernandes, G. L., Figueiredo, F., Hatushika, R. S., Leão, M. L., Mariano, B. A., Monteiro, B. A. A., de Oliveira, F. F., Panoutsos, T., Pires, J. P., Poppe, T. M., et al. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-024-01079-y · Cited by 12
Deep Learning for Contrast Enhanced Mammography - A Systematic Review
Sorin, V., Sklair‐Levy, M., Glicksberg, B. S., Konen, E., Nadkarni, G. N., & Klang, E. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.11.035 · Cited by 13
An Improved Grey Wolf Optimizer Based on Attention Mechanism for Solving Engineering Design Problems
Zhang, Y., Gao, Y., Huang, L., & Xie, X. (2024) · Symmetry · doi:10.3390/sym17010050 · Cited by 10
Sg-snn: a self-organizing spiking neural network based on temporal information
Gao, S., Zhu, R., Yu, Q., Tang, W., & Zhou, H. (2025) · Cognitive Neurodynamics · doi:10.1007/s11571-024-10199-6 · Cited by 10
“…Trans-former architecture relies entirely on the attention mechanism, with encoding and decoding processes dependent on stacked self-attention layers and point-wise fully connected layers, facilitating parallel processing of multimodal data(Niu et al. 2021; Guo et al. 2022; Vaswani et al. 2023).”
Nighttime light remote sensing image haze removal based on a deep learning model
Ma, X., Wang, Q., & Tong, X. (2025) · Remote Sensing of Environment · doi:10.1016/j.rse.2024.114575 · Cited by 10
Liang, J., & Cheng, J. (2025) · IEEE Access · doi:10.1109/access.2025.3528046 · Cited by 10
Heart Sound Classification Based on Multi-Scale Feature Fusion and Channel Attention Module
Li, M., He, Z., & Wang, H. (2025) · Bioengineering · doi:10.3390/bioengineering12030290 · Cited by 10
Tsai, Y., Tsai, C.-T., & Huang, J.-H. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07251-5 · Cited by 12
“By simplifying attention design, SimAM offers a novel 3D weight-free attention mechanism grounded in mathematical and neuroscience principles [16].”
Zhao, H., Zhang, C., Chen, J., Li, Z., Wang, F., & Gao, S. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3569726 · Cited by 11
“This mechanism includes several types such as channel attention [26], spatial attention [27], [28], [29], temporal attention [30], and branch attention [31], [32], achieving remarkable results across multiple visual tasks [33].”
Bi, Y., Huang, Y., Xuan, J., & Morota, G. (2025) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2025.104243 · Cited by 10
A Fitting Recognition Approach Combining Depth-Attention YOLOv5 and Prior Synthetic Dataset
Zhang, J., Lei, J. S., Qin, X., Li, B., Li, Z., Li, H., Zeng, Y., & Song, J. (2022) · Applied Sciences · doi:10.3390/app122111122 · Cited by 10
Soybean Seedling Root Segmentation Using Improved U-Net Network
Xu, X., Qiu, J., Zhang, W., Zhou, Z., & Ye, K. (2022) · Sensors · doi:10.3390/s22228904 · Cited by 10
“Inspired by the visual attention mechanism [31], the attention model was embedded in the downsampling and skip connection parts of the model, so that the model pays more attention to the root region and accurately identifies the feature of pseudo roots.”
MethodToward TR-PCB Bubble Detection via an Efficient Attention Segmentation Network and Dynamic Threshold
Zhou, J., Zhu, Q. X., Wang, Y., Zhou, X., Feng, M., Liu, X., & Mo, Y. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3244806 · Cited by 10
“The human visual system uses an attention mechanism to analyze and understand complex scenes efficiently and effectively [23].”
BackgroundAMFF-Net: An attention-based multi-scale feature fusion network for allergic pollen detection
Li, J., Wang, Q., Xiong, C., Zhao, L., Cheng, W., & Xu, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121158 · Cited by 10
CAU: A Causality Attention Unit for Spatial-Temporal Sequence Forecast
Qin, B., Meng, F., Yuan, S., & Mu, B. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3326289 · Cited by 10
“Macro Design: For the attention module, we have selected 3 candidates according to the review of [54]: CBAM [44], X-Linear [55], Triplet [45], Coordinate [56], and CoT [46].”
MethodPrivacy-Preserving Image Captioning with Deep Learning and Double Random Phase Encoding
Martin, A. D., Ahmadzadeh, E., & Moon, I. (2022) · Mathematics · doi:10.3390/math10162859 · Cited by 10
A Self-Attention Based Wasserstein Generative Adversarial Networks for Single Image Inpainting
Mao, Y., Zhang, T., Fu, B., & Thanh, D. N. H. (2022) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661822030245 · Cited by 10
“The human visual system employs attention to effectively comprehend complicated scenes [12] and is prone to pay more attention to vital regions while ignoring unnecessary ones.”
BackgroundYang, Y., Dong, J., Wang, Y., Yu, B., & Yang, Z. (2023) · Remote Sensing · doi:10.3390/rs15051328 · Cited by 10
Wang, Z., Li, J., Xu, Z., Yang, S., He, D., & Chan, S. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/8825587 · Cited by 9
Improvement of low-frequency ultrasonic image quality using a enhanced convolutional neural network
Lei, M., Zhang, W., Zhang, T., Wu, Y., Gao, D., Tao, X., Li, K., Shao, X., & Yang, Y. (2023) · Sensors and Actuators A Physical · doi:10.1016/j.sna.2023.114878 · Cited by 9
Liu, B., Ning, X., Ma, S., & Yang, Y. (2024) · Frontiers in Marine Science · doi:10.3389/fmars.2023.1328436 · Cited by 9
Multimodal Sentiment Analysis Method Based on Hierarchical Adaptive Feature Fusion Network
Zhang, H. (2024) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.335918 · Cited by 9
“Analyzing the sentiment information of users in multimodal data can better help cloud service providers and enterprises grasp the emotional state of users and guide their next steps (Kumar & Sivakumar, 2022; Guo et al., 2022).”
BackgroundSu, Q., & Mu, J. (2024) · Automation · doi:10.3390/automation5020011 · Cited by 10
Two-stage deep neural network for diagnosing fungal keratitis via in vivo confocal microscopy images
Li, C., Dai, W., Xiao, Y.-P., Qi, M., Zhang, L., Gao, L., Zhang, F., Lai, Y., Liu, C., Lu, J., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-68768-y · Cited by 9
“With the advances of attention mechanisms including become another popular deep networks for image analysis tasks, such as classification, segmentation, and object detection 24 .”
BackgroundXiong, J., Tang, M., Zong, L., Li, L., Hu, J., Bian, D., & Lv, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125078 · Cited by 9
Semantic Segmentation of Urban Remote Sensing Images Based on Deep Learning
Liu, J., Wu, J., Xie, H., Xiao, D., & Ran, M. (2024) · Applied Sciences · doi:10.3390/app14177499 · Cited by 9
Self-citationJi, W., Hu, L., Zhang, X., Lou, J., Chen, H., & Wang, Z. (2024) · Photonics · doi:10.3390/photonics11090794 · Cited by 9
Xie, F., Fan, Q., Li, G., Wang, Y., Sun, E., & Zhou, S. (2024) · Entropy · doi:10.3390/e26090810 · Cited by 9
“Its main role is to receive and process raw data for input into the network [18].”
BackgroundChavda, S., & Goyani, M. M. (2024) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.18.036511 · Cited by 9
“53,54 It can be applied to temporal, spatial, channel, and branch data spaces.”
Liu, Z., Zhou, X., Yang, H., Zhang, Q., Zhou, L., Wu, Y., Liu, Q., Yan, W., Song, J., Ding, M., et al. (2024) · Ultrasonics · doi:10.1016/j.ultras.2024.107486 · Cited by 12
Self-citationPeng, C., Sun, M., Zou, K., Zhang, B., Dai, G., & Tsoi, A. C. (2024) · Sensors · doi:10.3390/s24216912 · Cited by 9
“…regarded as a dynamic weight adjustment based on input image features, has been highly successful in various visual tasks, such as image classification, object detection, semantic segmentation, video comprehension, image generation, 3D vision, multimodal tasks, and self-supervised learning [29].”
BackgroundMS-YOLO: A Lightweight and High-Precision YOLO Model for Drowning Detection
Song, Q., Yao, B., Xue, Y., & Ji, S. (2024) · Sensors · doi:10.3390/s24216955 · Cited by 9
“In the context of drowning detection, what is crucial to reducing false positives and negatives lies in the relationship between the background and the target.”
BackgroundAfdhal, A., Saddami, K., Arief, M., Sugiarto, S., Fuadi, Z., & Nasaruddin, N. (2024) · IEEE Access · doi:10.1109/access.2024.3507713 · Cited by 12
“SE blocks are channels attention that offer better computational efficiency compared to other attention mechanisms or transformer architectures [69], making them more suitable for real-time object detection on edge devices with limited resources.”
Liu, Y., Liu, F., Tu, S., Liu, S., & Han, B. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104975 · Cited by 9
Zhang, H., Li, H., Sun, G., & Yang, F. (2025) · Animals · doi:10.3390/ani15020259 · Cited by 10
Detection of cervical cell based on multi-scale spatial information
Li, G., Fan, X., Xu, C., Lv, P., Wang, R., Ruan, Z., Zhou, Z., & Zhang, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-87165-7 · Cited by 10
“The method of shifting attention to the most important regions in an image while ignoring irrelevant parts is known as the attention mechanism 31 .”
Progressive Semantic Enhancement Network for Hyperspectral and LiDAR Classification
Fu, X., Zhou, X., Fu, Y., Liu, P., & Jia, S. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3513979 · Cited by 9
“The use of attention mechanisms has become increasingly crucial in image classification [32].”
Yuan, X., Yang, C., Chen, Y., & Zhang, Y. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3564932 · Cited by 9
“Inspired by the success of the attention mechanism [39], [40], we design the residual attention-based MNHU-Net for infrared small target feature extraction.”
Mamba-based hybrid network for multi-modality feature fusion
Wu, P., & Tang, J. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111064 · Cited by 9
Mozumder, M. S. A., Sakil, M. B. H., Hasan, M. R., Hasan, M. A., Fuad, K. M. N. R., Mridha, M. F., Islam, M. R., & Watanobe, Y. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3581950 · Cited by 9
“In fraud detection, attention-based models have been used to identify key transaction patterns, improving model performance and explainability [28].”
Wang, Z., Song, Y., Pang, L., Li, S., & Sun, G. (2025) · Sensors · doi:10.3390/s25134062 · Cited by 11
“It assigns corresponding weights, reducing the time information that LSTM needs to remember [14].”
YOLO-HVS: Infrared Small Target Detection Inspired by the Human Visual System
Wang, X., Sheng, Y., Hao, Q., Hou, H., & Nie, S. (2025) · Biomimetics · doi:10.3390/biomimetics10070451 · Cited by 9
“As exemplified by the attention mechanism, its design originates from the human visual system’s selective attention in information processing: humans tend to prioritize task-relevant information when dealing with abundant data [16].”
Ahmad, B., Li, Y., Wu, Z., Rehman, S. U., & Huang, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129021 · Cited by 10
LRFL-YOLO: A Large Receptive Field and Lightweight Model for Small Object Detection
Yan, Y., Wang, C., Zhou, X., Gao, X., & Qin, Z. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3595197 · Cited by 9
“However, the attention mechanism has several shortcomings, particularly its quadratic complexity [39], which limit its applicability.”
SAGAN: A Subgraph-Aware Graph Attention Network for Drug Repositioning
Meng, X., Wen, X., Sun, X., Xiang, J., Long, Y., Lin, X., & Li, M. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2025.9010108 · Cited by 10
A Survey of Small Sea-Surface Target Detection for Maritime Search and Rescue
Yin, J., Xu, G., Wang, N., Wang, N., & Zhang, Z. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3635199 · Cited by 9
“In computer vision, the attention mechanism can be seen as a dynamic selection process that prioritizes significant information by adaptively weighting features based on the input [126].”
Zhang, Q.-Q., Xu, Z.-L., Zou, Y.-T., & Li, D.-G. (2026) · Expert systems with applications · doi:10.1016/j.eswa.2026.132292 · Cited by 12
Attention Mechanism-based CNN for Surface Damage Detection of Wind Turbine Blades
Chen, Q., Liu, Z., & Lv, M. (2022) · International Conference on Machine Learning, Cloud Computing and Intelligent Mining (MLCCIM) · doi:10.1109/mlccim55934.2022.00061 · Cited by 9
“The attention mechanism is a way to dynamically adjust how important input image features are when making decisions, that is, to reduce the impact of unimportant information and increase attention to critical information [17].”
BackgroundSelf-citationLuo, Z., Qi, R., Li, Q., Zheng, J. J., & Shao, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28124-2_15 · Cited by 9
HPnet: Hybrid Parallel Network for Human Pose Estimation
Li, H., Yao, H., & Hou, Y. (2023) · Sensors · doi:10.3390/s23094425 · Cited by 10
“The attention module [30] aimed to model the significance of features and mainly focused on the channel dimension and spatial dimension.”
MethodResearch on Fraud Detection Method Based on Heterogeneous Graph Representation Learning
Zheng, X., Chen, F., Yin, Z., Zhang, J., & Shen, H. (2023) · Electronics · doi:10.3390/electronics12143070 · Cited by 9
FlowgateUNet: Dental CT image segmentation network based on FlowFormer and gated attention
Cao, D., Cai, B., & Liu, M. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02765-y · Cited by 9
“There have been many attempts to incorporate an attention mechanism in CNN to improve network performance [11].”
Guo, X., Gong, K., & Lu, C. (2023) · Coatings · doi:10.3390/coatings13122015 · Cited by 9
Two-Stream Networks for COPERT Correction Model with Time-Frequency Features Fusion
Xu, Z., Wang, R., Pan, K., Li, J., & Wu, Q. (2023) · Atmosphere · doi:10.3390/atmos14121766 · Cited by 9
Li, G., Zhao, S., Li, M., Zhou, M., & Ying, Z. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107677 · Cited by 9
Automated highway pavement crack recognition under complex environment
Zhang, Z., Yan, K., Zhang, X., Rong, X., Feng, D., & Yang, S. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26142 · Cited by 9
“AM has emerged as an essential component of neural network architecture within the realm of artificial intelligence and finds extensive applications in natural language processing [47], recommendation systems [48], speech recognition [49], computer vision [50], and other fields [51].”
BackgroundLong, D. T. (2024) · Cybernetics and Information Technologies · doi:10.2478/cait-2024-0010 · Cited by 9
Interpretable detector for cervical cytology using self-attention and cell origin group guidance
Jiang, P., Liu, J., Feng, J., Hua, C., Chen, Y., Li, C., Pang, B., & Cao, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108661 · Cited by 9
Self-citationChen, Y., Jiang, W., Wang, M., Kang, M., Weise, T., Wang, X., Tan, M., Xu, L., Li, X., & Zhang, C. (2022) · Remote Sensing · doi:10.3390/rs14246193 · Cited by 9
Human-machine interactions based on hand gesture recognition using deep learning methods
Zholshiyeva, L., Манбетова, Ж., Kaibassova, D., Kassymova, А., Tashenova, Z., Байжуманов, С., Yerzhanova, A., & Aikhynbay, K. (2023) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v14i1.pp741-748 · Cited by 10
“In recent years, deep learning has led to significant breakthroughs in pattern recognition [1], computer vision [2]–[4], and natural language processing ISSN: 2088-8708 Int J Elec & Comp Eng, Vol. 14 , No. 1, February 2024: 741-748 742 [5]–[8].”
BackgroundSangnark, S., Rattanachaisit, P., Patcharatrakul, T., & Vateekul, P. (2024) · IEEE Access · doi:10.1109/access.2024.3409077 · Cited by 8
“Attention mechanisms are novel techniques that are widely applied to deep learning models for various tasks of computer vision i.e. image classification, object detection, and image segmentation [35].”
BackgroundMethodWang, X., Wang, X., Zhang, M., Chun, T., Li, X., Sun, S., Wang, Y., Li, D., & Li, S. (2023) · Fire · doi:10.3390/fire6060237 · Cited by 8
“They can adaptively weight features according to the importance of the input, thus realizing a dynamic selection of features [35].”
BackgroundLao, Z., Han, Y., Ma, Y., & Chirikjian, G. S. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3293723 · Cited by 8
“As one of the most important concepts in the fields of deep neural networks, attention mechanisms are widely used in various applications [14], [20], [21].”
BackgroundDeep Learning-Based Automatic Helmet Recognition for Two-Wheeled Road Safety
Jakubec, M., Lieskovská, E., Brezani, A., & Tothova, J. (2023) · Transportation research procedia · doi:10.1016/j.trpro.2023.11.258 · Cited by 11
A method of particleboard surface defect detection and recognition based on deep learning
Zhang, C., Chun-ling, W., Zhao, L., Qu, X., & Gao, X. (2024) · Wood Material Science and Engineering · doi:10.1080/17480272.2024.2323579 · Cited by 8
Zhang, S., Liu, J., Bao, Z., Yang, Y., Wang, M., & Lin, Y. (2024) · IEEE Transactions on Reliability · doi:10.1109/tr.2024.3386216 · Cited by 9
“…AIequipitwiththeabilitytohandleintricatedataandtasks, renderingithighlyadvantageousinadiverserangeoffields, includingcomputervision[6],speechprocessing[7],hyper-spectralimagerecognition[8],spectrumresourceallocation[9], specificemitteridentification[10],[11],[12],[13],[14],and…”
Self-citationChen, H., & Kim, K. (2024) · IEEE Access · doi:10.1109/access.2024.3405957 · Cited by 8
“The primary purpose of introducing attention mechanisms in computer vision is to guide deep learning models to focus more on specific parts of the input data, thereby processing a large amount of visual information more effectively [35], [36].”
BackgroundDAENet: Deformable Attention Edge Network for Automatic Coastline Extraction from Satellite Imagery
Kang, B., Wu, J., Xu, J., & Wu, C. (2024) · Remote Sensing · doi:10.3390/rs16122076 · Cited by 8
Improvement Methods for Medical Image Segmentation Based on Attention U-Net Networks
Shen, F. (2024) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit61980.2024.10581570 · Cited by 8
A self-attention armed optronic transformer in imaging through scattering media
Huang, Z., Shi, M., Ma, J., Gao, Y., & Liu, X. (2024) · Optics Communications · doi:10.1016/j.optcom.2024.130943 · Cited by 8
Cubixel: a novel paradigm in image processing using three-dimensional pixel representation
Aburass, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20081-6 · Cited by 8
ExGAT: Context extended graph attention neural network
Quan, P., Zheng, L., Zhang, W., Xiao, Y., Niu, L., & Shi, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106784 · Cited by 8
Ambreen, S., Iqbal, M., Asghar, M. Z., Mazhar, T., Khattak, U. F., Khan, M. A., & Hamam, H. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01356-0 · Cited by 8
“The implementation and design of the (FLC) fuzzy logic system, including its membership functions and fuzzy rules, are described in detail (Guo et al. 2022). iii.”
Steering-Angle Prediction and Controller Design Based on Improved YOLOv5 for Steering-by-Wire System
Ye, C., Wang, Y., Wang, Y., & Liu, Y. (2024) · Sensors · doi:10.3390/s24217035 · Cited by 8
“The attention mechanism is inspired by humans’ ability to process external information, where individuals selectively attend to relevant information while filtering out irrelevant stimuli due to the limited processing capacity of the human brain [37,38].”
BackgroundDynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis
Wang, N., & Wang, Q. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3702996 · Cited by 9
Zhang, J., Yu, J., Wei, X., Zhou, K., Niu, W., Wei, Y., Zhao, C., Chen, G., Jin, F., & Song, K. (2024) · Materials Characterization · doi:10.1016/j.matchar.2024.114532 · Cited by 8
CFDformer: Medical image segmentation based on cross fusion dual attention network
Yang, Z., Wang, H., Liu, Y., & Zhang, F. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107208 · Cited by 8
DACA-Net: Detail-aware network with contrast attention for locating liquid crystal display defects
Luo, S., Chen, H., & Liu, B. (2024) · Displays · doi:10.1016/j.displa.2024.102913 · Cited by 8
Wang, S., Cheng, D.-J., Fang, X., & Zhang, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126385 · Cited by 9
Khalifa, A. B., Mili, M., Maâtouk, M., Abdallah, A. B., Abdelaali, M., Gaied, S., Ali, A. B., Lahouel, Y., Bedoui, M. H., & Zrig, A. (2025) · Diagnostics · doi:10.3390/diagnostics15020207 · Cited by 8
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
