The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 101–200 of 1,001
Eftekhari, A., Samadzadegan, F., & Javan, F. D. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103180 · Cited by 59
An external attention-based feature ranker for large-scale feature selection
Xue, Y., Zhang, C., Neri, F., Gabbouj, M., & Zhang, Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111084 · Cited by 59
Attention based hybrid deep learning model for wearable based stress recognition
Tanwar, R., Phukan, O. C., Singh, G., Pal, P. K., & Tiwari, S. M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107391 · Cited by 68
Zhao, S., Kang, F., & Li, J. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105145 · Cited by 58
Sahoo, J. P., Sahoo, S. P., Ari, S., & Patra, S. K. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3246488 · Cited by 57
“In CNN architecture, attention modules refine the FMs and focus on discriminative regions rather than learning irrelevant background information to enhance recognition performance [23].”
BackgroundArya, D., Maeda, H., & Sekimoto, Y. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102388 · Cited by 61
An Interpretable Parallel Spatial CNN-LSTM Architecture for Fault Diagnosis in Rotating Machinery
Zhou, Q., & Tang, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3422969 · Cited by 57
“This technique combines the strengths of the traditional LSTM framework with the power of a self-attention mechanism [25], [26].”
MethodBrain tumor segmentation in multimodal MRI via pixel-level and feature-level image fusion
Liu, Y., Mu, F., Shi, Y., Cheng, J., Li, C., & Chen, X. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.1000587 · Cited by 56
“According to a recent survey on attention mechanism (Guo et al., 2022), the attention mechanism used in our MSFF module belongs to the branch attention, which can be viewed as a dynamic branch selection mechanism and typically used in a multi-branch architecture.”
MethodEfficient attention-based CNN network (EANet) for multi-class maize crop disease classification
Albahli, S., & Masood, M. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.1003152 · Cited by 63
“Recent studies show that the attention method can be supplemented with CNN to obtain discriminative features of the region of interest (Guo et al., 2022).”
BackgroundMethodInfluentialLarge-Kernel Attention for 3D Medical Image Segmentation
Li, H., Yang, N., Ser, J. D., & Yang, G. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10126-7 · Cited by 56
“Thus, adaptation in channel maps is important for attention to build dependencies within channels [12, 14, 15].”
BackgroundMethodVisual Concrete Bridge Defect Classification and Detection Using Deep Learning: A Systematic Review
Amirkhani, D., Allili, M. S., Hebbache, L., Hammouche, N., & Lapointe, J. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3365296 · Cited by 66
“2) Attention-Based Models: Attention-augmented CNN models aim at incorporating mechanisms that put more emphasis on relevant CNN channels or image locations to achieve better defect classification and detection performance [147].”
MethodHu, H., Mei, Y., Wei, Y., Xu, Z., Zhao, Y., Xu, H., Mao, X., & Huang, L. (2024) · LWT · doi:10.1016/j.lwt.2024.116436 · Cited by 56
Swin-fusion: An adaptive multi-source information fusion framework for enhanced tool wear monitoring
Hou, K., Li, R., Liu, X., Yue, C., Wang, Y., Liu, X., & Xia, W. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.02.003 · Cited by 59
Yang, S., Jin, A., Zeng, X., Wang, H., Hong, X., & Lei, M. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.107983 · Cited by 55
Xia, K., Lv, Z., Zhou, C., Gu, G., Zhao, Z., Liu, K., & Li, Z. (2023) · Sensors · doi:10.3390/s23115114 · Cited by 58
“Deep learning-based detection methods can achieve certain effects in addressing the above detection challenges through targeted algorithms, such as data augmentation [5], multi-scale feature fusion [6], and attention mechanisms [7,8].”
BackgroundWang, H., Chong, K. K. L., Lin, Z., Yu, X., & Pan, Y. F. (2023) · Electronics · doi:10.3390/electronics12122697 · Cited by 55
Underwater-YCC: Underwater Target Detection Optimization Algorithm Based on YOLOv7
Chen, X., Yuan, M., Yang, Q. Z., Yao, H., & Wang, H. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11050995 · Cited by 54
“…a tt ention mechanism can be regarded as a process of dynamic weight adjustment based on the features of the input image around the target position [22], so that the machine focuses on the target to be detected and recognized as much as possible, and optimizes the allocation of computing resources…”
BackgroundNovel joint transfer fine-grained metric network for cross-domain few-shot fault diagnosis
Hu, J., Li, W., Wu, A., & Tian, Z. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110958 · Cited by 54
DSW-YOLOv8n: A New Underwater Target Detection Algorithm Based on Improved YOLOv8n
Liu, Q., Huang, W. D., Duan, X., Wei, J., Hu, T., Yu, J., & Huang, J. (2023) · Electronics · doi:10.3390/electronics12183892 · Cited by 59
Recursive-NeRF: An Efficient and Dynamically Growing NeRF
Yang, G., Zhou, W., Peng, H.-Y., Liang, D., Mu, T., & Hu, S. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3204608 · Cited by 53
“Transformer has made great progress in the field of vision recently [30], [31], [32], [33].”
BackgroundSelf-citationInvestigating attention mechanisms for plant disease identification in challenging environments
Duhan, S., Gulia, P., Gill, N. S., Shukla, P. K., Khan, S. B., Almusharraf, A. I., & Alkhaldi, N. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e29802 · Cited by 53
“For instance, while spatial attention is necessary for tasks like object detection and semantic segmentation, channel attention is required for image classification [3,87].”
BackgroundMethodAcoustic scene classification: A comprehensive survey
Ding, B., Zhang, T., Wang, C., Liu, G., Liang, J., Hu, R., Wu, Y., & Guo, D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121902 · Cited by 57
Toledo, C. A., Crawford, M. M., & Tuinstra, M. R. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1408047 · Cited by 63
“Use of advanced predictive models has signi fi cantly altered the approach of researchers toward the development of new crop varieties in maize breeding experiments (Xu et al., 2022).”
BackgroundQuantum Neural Network Classifiers: A Tutorial
Li, W., Lu, Z., & Deng, D.-L. (2022) · SciPost Physics Lecture Notes · doi:10.21468/scipostphyslectnotes.61 · Cited by 58
“More recently, the attention mechanism has been playing an important role in the field of both computer vision [29, 30] and natural language processing [31, 32], which ignites tremendous interest in exploring powerful and practical deep learning models.”
BackgroundVachmanus, S., Noraset, T., Piyanonpong, W., Rattananukrom, T., & Tuarob, S. (2023) · IEEE Access · doi:10.1109/access.2023.3345225 · Cited by 53
“Another scheme of fusing metadata into visual information is using metadata to guide the image feature extraction without changing the dimension of visual feature maps, similar to the attention mechanism [58].”
MethodDetection of Surface Defects on Railway Tracks Based on Deep Learning
Wang, M., Li, K., Zhu, X., & Zhao, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3224594 · Cited by 50
Jiménez-Mesa, C., Arco, J. E., Martínez-Murcia, F. J., Suckling, J., Ramı́rez, J., & Górriz, J. M. (2023) · Pharmacological Research · doi:10.1016/j.phrs.2023.106984 · Cited by 62
“An emerging trend in DL models is the utilization of attention-based models [65], which is gaining increasing popularity.”
Myint, P. Y. W., Lo, S. L., & Zhang, Y. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103695 · Cited by 53
Liu, Y., He, G., Wang, Z., Li, W., & Huang, H. (2022) · Sensors · doi:10.3390/s22134953 · Cited by 49
“Attention mechanisms can be basically divided into four types [31], and two of them are broadly utilized in object detection: • Channel attention, which focuses on important objects, such as SENet [32]; • Spatial attention, which focuses on important locations, such as self-attention in Transformer and DCN [33]; The c…”
BackgroundNi, Y., Liu, J., Chi, W., Wang, X., & Li, D. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3379398 · Cited by 49
“By incorporating the attention mechanism into the network [33], [34], [35], [36] architecture for global context information extraction, the researchers get rid of the limitation of focusing on the local area [22].”
BackgroundSelf-citationJain, E., Kaushik, P., Kukreja, V., Ati, M., Hariharan, S., Ahuja, V., Bhattacherjee, A., & Kaushal, R. K. (2025) · IEEE Access · doi:10.1109/access.2025.3563256 · Cited by 49
“This approach allows healthcare facilities with diverse patient populations to collectively improve diagnostic models, thus providing a scalable, privacy-friendly solution to enhancing diagnostic accuracy across healthcare settings [46], [47], [48] as depicted in Fig 7.”
Guo, Y., He, W. P., Xia, Y., & Zhang, H. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.10.014 · Cited by 48
Wei, P., Li, R., Liu, X., Gao, H., Dai, M., Zhang, Y., Zhao, W., & Liu, E. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2024.102741 · Cited by 50
Brain-Inspired Learning, Perception, and Cognition: A Comprehensive Review
Jiao, L., Ma, M., He, P., Geng, X., Liu, X., Liu, F., Ma, W., Yang, S., Hou, B., & Tang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3401711 · Cited by 54
“Only a portion of the information can be screened and processed by selective attention, allowing it to enter consciousness [29].”
Jin, A., & Zeng, X. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11010069 · Cited by 47
He, J., Xu, Y., Pan, Y. F., & Wang, Y. (2024) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2024.111288 · Cited by 46
Vidivelli, S., Manikandan, R., & Dharunbalaji, A. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.054360 · Cited by 54
A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition
Wang, H., Kang, P., Gao, Q., Jiang, S., & Shull, P. B. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3194017 · Cited by 45
“Inspired by the human visual system, the attention mechanism was designed and introduced into deep learning to efficiently and effectively capture salient regions [44].”
MethodLiu, Y., Cao, Z., Xiong, H., Du, J., Cao, H., & Zhang, L. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3241801 · Cited by 45
“1) Attention Module: An attention module is designed inspired by [30], aiming to enable the policy network to focus on information that contributes to the determination of the target velocity of a CDPR with MBs.”
BackgroundA survey of deep learning-based 3D shape generation
Xu, Q., Mu, T., & Yang, Y. (2023) · Computational Visual Media · doi:10.1007/s41095-022-0321-5 · Cited by 45
“Concurrently, with the explosive growth of data [2, 3] and the significantly enhanced power of modern computational devices [4], deep learning based on large-scale neural networks has become an active research area, and a fundamental technique in computer vision and computer graphics [5, 6].”
BackgroundSelf-citationResearch on Underwater Small Target Detection Algorithm Based on Improved YOLOv7
Yi, W., & Wang, B. (2023) · IEEE Access · doi:10.1109/access.2023.3290903 · Cited by 46
“The issue of small target features not being obvious can be resolved by introducing the attention mechanism because the channel attention mechanism [33] automatically learns the importance of each channel, boosting useful features and suppressing features that are of little use to the task at hand.”
BackgroundWang, Z., Li, L., Zeng, C., & Yao, J. (2023) · Sensors · doi:10.3390/s23198190 · Cited by 45
“Within the scope of computer vision, attention mechanisms are strategically deployed for tasks encompassing image classification, target recognition, and image generation, culminating in augmented model performance by virtue of their capacity to concentrate on pertinent image regions [44].”
BackgroundSSCNet: A Spectrum-Space Collaborative Network for Semantic Segmentation of Remote Sensing Images
Li, X., Xu, F., Yong, X., Chen, D., Xia, R., Ye, B., Gao, H., Chen, Z., & Lyu, X. (2023) · Remote Sensing · doi:10.3390/rs15235610 · Cited by 45
Gao, Z., Wang, Y., Li, X., & Yao, J. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad53f1 · Cited by 45
Axford, D., Sohel, F. A., Vanderklift, M. A., & Hodgson, A. J. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102842 · Cited by 50
Fusing Bone-Conduction and Air-Conduction Sensors for Complex-Domain Speech Enhancement
Wang, H., Zhang, X., & Wang, D. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3209943 · Cited by 44
“Since then, it has been widely employed in tasks like automatic speech recognition [25], natural language processing [5] and computer vision [9].”
MethodImproved domain adaptive rice disease image recognition based on a novel attention mechanism
Chen, L., Zou, J., Yuan, Y., & He, H. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.107806 · Cited by 47
Liu, P., Wang, Q., Zhang, H., Mi, J., & Liu, Y. (2023) · Remote Sensing · doi:10.3390/rs15092429 · Cited by 48
A real-time and accurate convolutional neural network for fabric defect detection
Li, X., & Zhu, Y. T. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01317-8 · Cited by 51
Yao, X., Lu, X., Jiang, Q., Shen, Y., Xu, F., & Zhu, Q. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102560 · Cited by 44
EEG motor imagery decoding: a framework for comparative analysis with channel attention mechanisms
Wimpff, M., Gizzi, L., Zerfowski, J., & Yang, B. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad48b9 · Cited by 44
“The deep learning community has tried to include different kinds of attention mechanisms for many years (see [32] for a review).”
LIDD-YOLO: a lightweight industrial defect detection network
Luo, S., Xu, Y., Zhang, C., Jin, J., Kong, C., Xu, Z., Guo, B., Tang, D., & Cao, Y. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad9d65 · Cited by 49
Razali, N., Arbaiy, N. B., Lin, P., & Ismail, S. (2025) · Electronics · doi:10.3390/electronics14040705 · Cited by 44
Improved Algorithm for Insulator and Its Defect Detection Based on YOLOX
Han, G., Li, T., Li, Q., Zhao, F., Zhang, M., Wang, R., Yuan, Q., Liu, K., & Qin, L. (2022) · Sensors · doi:10.3390/s22166186 · Cited by 46
“The process of feature fusion cannot effectively reduce the impact of redundant information on the detection capability of the model, while the attention mechanism can as ign greater weight to importan f atures, duce the weight of redundant information, and lighten the influence of redundant features [30].”
BackgroundAttention-guided dynamic multi-branch neural network for underwater image enhancement
Yan, X., Qin, W., Wang, Y., Wang, G., & Fu, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110041 · Cited by 43
Agac, S., & İncel, Ö. D. (2023) · Diagnostics · doi:10.3390/diagnostics13111861 · Cited by 51
Transformer-Based Under-Sampled Single-Pixel Imaging
Tian, Y., Fu, Y., & Zhang, J. (2023) · Chinese Journal of Electronics · doi:10.23919/cje.2022.00.284 · Cited by 43
Wang, N., Shi, D., Li, Z., Chen, P., & Ren, X. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102557 · Cited by 43
Baroni, G. L., Rasotto, L., Roitero, K., Tulisso, A., Loreto, C. D., & Mea, V. D. (2024) · Journal of Imaging · doi:10.3390/jimaging10050108 · Cited by 43
“In this research, the ViT model serves as the foundational architecture for all our experiments, owing to its established success in diverse computer vision tasks, especially image classification [8,45,46].”
MethodTransferable Multimodal Attack on Vision-Language Pre-training Models
Wang, H., Dong, K., Zhu, Z., Qin, H., Liu, A., Fang, X., Wang, J., & Liu, X. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00102 · Cited by 53
“Specifically, for the extraction of attention features, some studies have indicated that the information in both lower and higher layers of the attention module is a mixture of local and global information [38], [39].”
BackgroundRice seedling row detection based on morphological anchor points of rice stems
Li, D., Li, B., Long, S., Feng, H., Xi, T., Kang, S., & Wang, J. (2023) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2022.12.012 · Cited by 42
CO 3 Net: Coordinate-Aware Contrastive Competitive Neural Network for Palmprint Recognition
Yang, Z., Xia, W., Qiao, Y., Lu, Z., Zhang, B., Leng, L., & Zhang, Y. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3276506 · Cited by 43
“The attention mechanism is introduced into computer vision to imitate the human visual system which always pays more attention to salient targets [41].”
BackgroundMachine learning for drug repositioning: Recent advances and challenges
Cai, L., Chu, J., Xu, J., Meng, Y., Lu, C., Tang, X., Wang, G., Tian, G., & Yang, J. (2023) · Current Research in Chemical Biology · doi:10.1016/j.crchbi.2023.100042 · Cited by 43
Huang, W., Gao, T., Hao, Y., & Wang, X. (2023) · Energy Economics · doi:10.1016/j.eneco.2023.107106 · Cited by 43
A fusion of VGG-16 and ViT models for improving bone tumor classification in computed tomography
Chen, W., Ayoub, M., Liao, M., Shi, R., Zhang, M., Su, F., Huang, Z., Li, Y., Wáng, Y., & Wong, K. K. L. (2023) · Journal of bone oncology · doi:10.1016/j.jbo.2023.100508 · Cited by 46
“Guo et al., [24] provides many ViT models, including ViT-B /32, ViT-B /16, ViT-L /32, ViT-L /16 and ViT-H /14.”
BackgroundResultInfluentialPrompting large language model with context and pre-answer for knowledge-based VQA
Hu, Z., Yang, P., Jiang, Y., & Bai, Z. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110399 · Cited by 45
Liang, T., Jin, Y., Liu, W., Wang, T., Feng, S., & Li, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3377252 · Cited by 45
“Restrictions apply. purposes [80].”
Marjani, M., Mahdianpari, M., Mohammadimanesh, F., & Gill, E. W. (2024) · Remote Sensing · doi:10.3390/rs16132427 · Cited by 42
“Channel A tt this study, we employed CA, SA, and a ViT model that incorporates multi-head self-attention mechanism (MHSeA). ention (CA)”
A novel method based on CNN-BiGRU and AM model for bearing fault diagnosis
Xu, Z., Li, Y., Huang, H., Deng, Z., & Huang, Z. (2024) · Journal of Mechanical Science and Technology · doi:10.1007/s12206-024-0610-2 · Cited by 42
“Inspired by the attention mechanism of the human brain, a similar mechanism has been proposed in deep learning literature [23].”
Yang, F., Zuo, R., Xiong, Y., Wang, J., & Zhang, G. (2023) · Journal of Geochemical Exploration · doi:10.1016/j.gexplo.2023.107274 · Cited by 44
Yang, R., Wang, M., Zhang, Z., Liu, Z., Li, M., & Lin, B. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107800 · Cited by 44
Self-citationStacked CNN-based multichannel attention networks for Alzheimer disease detection
Hassan, N., Miah, A. S. M., Suzuki, K., Okuyama, Y., & Shin, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-85703-x · Cited by 43
YOLOv7-Plum: Advancing Plum Fruit Detection in Natural Environments with Deep Learning
Tang, R., Lei, Y., Luo, B., Zhang, J., & Mu, J. (2023) · Plants · doi:10.3390/plants12152883 · Cited by 40
“The introduction of an attention mechanism can make the network focus on the target area, enhance the effective feature extraction ability of the network, reduce the interference of invalid targets, and obtain more detailed information [41,42].”
BackgroundResearch on surface defect detection algorithm of pipeline weld based on YOLOv7
Xu, X., & Li, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-52451-3 · Cited by 47
Holographic communication using programmable coding metasurface
Zhang, F., Wang, C., Feng, W., Liu, T., Wang, Z., Wang, Y., Wang, M., & Xu, H. (2024) · Nanophotonics · doi:10.1515/nanoph-2023-0925 · Cited by 42
Zubair, M., Owais, M., Hassan, T., Bendechache, M., Hussain, M., Hussain, I., & Werghi, N. (2025) · Scientific Reports · doi:10.1038/s41598-025-97256-0 · Cited by 61
“Attention techniques often take the form of plug-and-play attention modules that may enhance a block’s convolutional outputs and help the entire network learn more illuminating information 55 .”
Zhou, Y., Ji, A., Zhang, L., & Xue, X. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105554 · Cited by 41
Wheel hub defect detection based on the DS-Cascade RCNN
Cheng, S., Lu, J., Yang, M., Zhang, S., Xu, Y., Zhang, D., & Wang, H. (2022) · Measurement · doi:10.1016/j.measurement.2022.112208 · Cited by 40
Self-citationVideo restoration based on deep learning: a comprehensive survey
Rota, C., Buzzelli, M., Bianco, S., & Schettini, R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10302-5 · Cited by 39
“Typical attention types are: (i) channel attention, used to select the most important channels; (ii) spatial attention, used to select the most important regions; (iii) temporal attention, used to select the most important frames (Guo et al. 2022).”
BackgroundMa, Z., Huo, Q. C., Wang, W., & Zhang, T. (2023) · Energy · doi:10.1016/j.energy.2023.127747 · Cited by 39
Understanding Self-attention Mechanism via Dynamical System Perspective
Huang, Z., Liang, M., Qin, J., Zhong, S., & Lin, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00136 · Cited by 44
“The self-attention mechanism (SAM) [41, 15, 16, 9, 43, 4] is widely used in various artificial intelligence fields and has successfully improved the models’ performance in a number of vision tasks, including image classification [22, 60, 47], object detection [37, 56, 25], instance seg-* co-first…”
Xu, Y., Qiao, W., Zhao, J., Zhang, Q., & Li, H. (2023) · Earthquake Engineering and Engineering Vibration · doi:10.1007/s11803-023-2153-4 · Cited by 38
“Guo et al. (2020) proposed a decision algorithm to automatically learn where to share or branch in a network.”
BackgroundMethodDensePPMUNet-a: A Robust Deep Learning Network for Segmenting Water Bodies From Aerial Images
Xiang, D.-Q., Zhang, X., Wu, W., & Liu, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3251659 · Cited by 39
“More detailed information could be found in [45].”
BackgroundKommanduri, R., & Ghorai, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127444 · Cited by 38
Adaptive Compression Offloading and Resource Allocation for Edge Vision Computing
Xiao, W., Hao, Y., Liang, J., Hu, L., AlQahtani, S. A., & Chen, M. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3400820 · Cited by 38
“Therefore, we employ the discriminative localization matrices of the gradient-weighted class activation mapping (Grad-CAM) [28] to generate the Q matrix.”
Novel approach to quantitative risk assessment of reservoir landslides using a hybrid CNN-LSTM model
Wang, L., Yang, K., Wu, C., Zhou, Y., Liu, J., & Hu, H. (2024) · Landslides · doi:10.1007/s10346-024-02398-3 · Cited by 42
“…address geotechnical-related issues in recent years by virtue of its excellent performance in data mining and learning analytics (e.g., Wei et al. 2021; Guo et al. 2022a; Soranzo et al. 2023; He et al. 2023b; Nava et al. 2023; Wang et al. 2023; Wu et al. 2023; Feng et al. 2024; Wang et al. 2024).”
Self-citationSun, Z., Meng, C., Cheng, J., Zhang, A., & Chang, S. (2022) · Remote Sensing · doi:10.3390/rs14246312 · Cited by 41
Automating Building Damage Reconnaissance to Optimize Drone Mission Planning for Disaster Response
Hu, D., Li, S., Du, J., & Cai, J. (2023) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0001061 · Cited by 38
Wang, Z., Zhang, X., Fu, R., Wang, D., Chen, X., & Wang, H. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3265108 · Cited by 37
“For a comprehensive survey of attention in CV, see [45].”
BackgroundBlood Cell Detection Method Based on Improved YOLOv5
Guo, Y., & Zhang, M. (2023) · IEEE Access · doi:10.1109/access.2023.3290905 · Cited by 37
“Therefore, in order to improve the detection accuracy of blood cells, Attention Mechanism [17] is introduced into the YOLOv5 structure to better extract the image features of different blood cells.”
MethodLSKANet: Long Strip Kernel Attention Network for Robotic Surgical Scene Segmentation
Liu, M., Han, Y., Wang, J., Wang, C.-H., Wang, Y., & Meijering, E. H. W. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3335406 · Cited by 38
“Attention mechanisms can be treated as a dynamic selection process that is realized by adjusting weighting features according to the importance of the input, which is widely used in feature extraction [36].”
BackgroundModeling the Relative Visual Tempo for Self-supervised Skeleton-based Action Recognition
Zhu, Y., Han, H., Yu, Z., & Liu, G. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.01279 · Cited by 41
“Benefiting from recent attention mechanisms [5, 13, 44], we try to emphasize these connections by designing a novel module, FSEM, which contains an Intra-Inter-Part block (IIPB) for local spatial modeling and a Non-local block (NLB) [32] for global spatiotemporal modeling.”
MethodHybrid Attention Fusion Embedded in Transformer for Remote Sensing Image Semantic Segmentation
Chen, Y., Dong, Q., Wang, X., Zhang, Q., Kang, M., Jiang, W., Wang, M., Xu, L., & Zhang, C. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3358851 · Cited by 37
“Other works specifically design attention mechanisms targeting spe-cific problems and challenges to effectively alleviate the impact of noise on segmentation [22].”
BackgroundLv, S., Liang, T., Zhang, K., Jiang, S., Ouyang, B., Li, Q., & Li, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125727 · Cited by 40
Wei, D., Wei, X., Tang, Q., Jia, L., Yin, X., & Ji, Y. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105822 · Cited by 37
Automatic cardiac arrhythmias classification using CNN and attention‐based RNN network
Sun, J. (2023) · Healthcare Technology Letters · doi:10.1049/htl2.12045 · Cited by 37
“Attention mechanisms are a well-known technique in computer vision [17] and natural language processing [18].”
BackgroundBhagabati, B., Sarma, K. K., & Bora, K. C. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102398 · Cited by 39
Explainability and Evaluation of Vision Transformers: An In-Depth Experimental Study
Stassin, S., Corduant, V., Mahmoudi, S. A., & Siebert, X. (2023) · Electronics · doi:10.3390/electronics13010175 · Cited by 39
“Guo et al. [26] have demonstrated that our brains indeed concentrate on specific regions of an image when identifying an object.”
Wu, Z., Ma, N., Wang, C., Xu, C., Xu, G., & Li, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110427 · Cited by 39
Hybrid attention network for citrus disease identification
Zhang, F., Jin, X., Lin, G., Jiang, J. S., Wang, M., An, S., Hu, J., & Lyu, Q. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.108907 · Cited by 36
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
