The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 601–700 of 1,001
Li, X., Qiu, H., Ding, A., & Fan, P. (2024) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2024.136729 · Cited by 13
Shang, L., Zhang, Y., Deng, Y., & Wang, D. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3533919 · Cited by 13
“…between the presentation of misleading claims and their debunking or satirical use, context awareness techniques (e.g., attention mechanism [77], hidden Markov models [78]) could be integrated into MultiTec to effectively capture the context dependency between the mis-leading claims and…”
Vision-Based Collision Warning Systems with Deep Learning: A Systematic Review
Chitraranjan, C. D., Vipulananthan, V., & Sritharan, T. (2025) · Journal of Imaging · doi:10.3390/jimaging11020064 · Cited by 16
“We refer interested readers to [66] for a comprehensive survey on attention mechanisms in computer vision.”
Deep Learning-Based Algorithm for Road Defect Detection
Li, S., & Zhang, D. (2025) · Sensors · doi:10.3390/s25051287 · Cited by 13
Multi-modal Language models in bioacoustics with zero-shot transfer: a case study
Miao, Z., Elizalde, B., Deshmukh, S., Kitzes, J. A., Wang, H., Dodhia, R. M., & Ferres, J. L. (2025) · Scientific Reports · doi:10.1038/s41598-025-89153-3 · Cited by 25
“We didn’t choose larger deep learning models because it has been reported that deeper models, such as ResNet-50 78 , Inception 79 , and Vision Transformers 27,28 , tend to overfit more easily on smaller datasets and their performance gains are often limited 80 .”
Bahuguna, A., Govil, M. C., & Bhaumik, G. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04071-1 · Cited by 14
Self-supervised coarse-to-fine monocular depth estimation using a lightweight attention module
Li, Y., Luo, F., & Xiao, C. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0279-3 · Cited by 13
“Attention mechanisms have achieved great success in many visual tasks, such as image classification, object detection, and semantic segmentation [18].”
BackgroundZhang, H., Qiao, G., Liu, S., Lyu, Y., Yao, L., & Ge, Z. (2022) · Coloration Technology · doi:10.1111/cote.12644 · Cited by 14
Xie, Y., Chen, R., Yu, M., Rui, X., & Du, X. (2023) · International Journal of Remote Sensing · doi:10.1080/01431161.2023.2229498 · Cited by 13
“The attention mechanism can be used for the extraction of global features from high-resolution remote sensing images to highlight the locations of significant changes in the image while weakening the locations with high similarity (Guo et al. 2022).”
DeepTPA-Net: A Deep Triple Attention Network for sEMG-Based Hand Gesture Recognition
Li, Q., Luo, Z., Qi, R., & Zheng, J. J. (2023) · IEEE Access · doi:10.1109/access.2023.3312219 · Cited by 13
“groups, as outlined in [32]: (1) temporal attention, (2) spatial attention, (3) channel attention, (4) branch attention, (5) channel & spatial attention, and (6) spatial & temporal”
BackgroundMethodResearch on Railway Track Extraction Method Based on Edge Detection and Attention Mechanism
Weng, Y., Huang, X., Chen, X., He, J., Li, Z., & Yi, H. (2024) · IEEE Access · doi:10.1109/access.2024.3366184 · Cited by 13
“With the development of attention mechanisms, more and more attention mechanisms have been applied in image processing [32].”
BackgroundChen, X., He, S., Paik, K., Wong, Y. H., & Zhang, S. (2025) · Journal of Materials Informatics · doi:10.20517/jmi.2025.26 · Cited by 14
Self-citationNi, Y., Xue, D., Chi, W., Luan, J., & Liu, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103786 · Cited by 12
Self-citationTowards Explainable Deep Learning in Computational Neuroscience: Visual and Clinical Applications
Mehmood, A., Mehmood, F., & Kim, J. (2025) · Mathematics · doi:10.3390/math13203286 · Cited by 14
Frequency-Enhanced Channel-Spatial Attention Module for Grain Pests Classification
Yu, J., Shen, Y., Liu, N., & Pan, Q. (2022) · Agriculture · doi:10.3390/agriculture12122046 · Cited by 12
“[31] surveyed attention models in deep neural networks and encouraged various studies to improve deep learning results by using attention mechanisms.”
BackgroundAttention Guided Policy Optimization for 3D Medical Image Registration
Hu, J., Shuai, Z., Wang, X., Hu, S., Sun, S., Lyu, S., & Wu, X. (2023) · IEEE Access · doi:10.1109/access.2023.3264476 · Cited by 12
“In recent years, attention mechanisms in computer vision problems are increasingly used [28], [29].”
BackgroundSelf-citationSiamHAS: Siamese Tracker with Hierarchical Attention Strategy for Aerial Tracking
Liu, F., Liu, J., Chen, Q., Wang, X., & Liu, C. (2023) · Micromachines · doi:10.3390/mi14040893 · Cited by 13
Self-citationLi, F., Zhou, Y., Chen, Y., Li, J., Dong, Z., & Tan, M. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01185-2 · Cited by 12
Lymphocyte detection for cancer analysis using a novel fusion block based channel boosted CNN
Rauf, Z., Khan, A. R., Sohail, A., Alquhayz, H., Gwak, J., & Khan, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-40581-z · Cited by 12
Attention-Based Human Age Estimation from Face Images to Enhance Public Security
Rahman, M. A., Aonty, S. S., Deb, K., & Sarker, I. H. (2023) · Data · doi:10.3390/data8100145 · Cited by 14
“Different attention modules are already used in computer vision for image classification, recognition, and segmentation [11].”
BackgroundMethodRecognizing materials in cultural relic images using computer vision and attention mechanism
Pei, H., Zhang, C., Zhang, X., Liu, X., & Ma, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122399 · Cited by 12
Zhao, Q., Wang, X., Wang, B., Wang, L., Liu, W., & Li, S. (2023) · Remote Sensing · doi:10.3390/rs15235492 · Cited by 12
“Attention mechanisms aim to imitate the human visual system that can capture the salient regions in complex scenes [34].”
BackgroundLiu, W., Li, G., Huang, Z., Jiang, W., Luo, X., & Xu, X. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1310323 · Cited by 14
“Attention mechanism is a technology that emulates human attention processes, which has grown in significance within the domains of natural language processing and deep learning in recent years (46).”
BackgroundFan, X., Liu, H., Zheng, H., Zhai, J., & Zhang, L. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105880 · Cited by 12
Edge-guided adaptive multi-feature learning for breast ultrasound image segmentation
Bai, X., Ma, Y., Wang, W., Gao, X., & Xue, Z. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106076 · Cited by 12
Ensemble Learning based on CNN and Transformer Models for Leaf Diseases Classification
Li, L.-H., & Tanone, R. (2024) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom60618.2024.10418393 · Cited by 14
A New Method to Detect Buffalo Mastitis Using Udder Ultrasonography Based on Deep Learning Network
Zhang, X., Li, Y., Zhang, Y., Yao, Z., Zou, W., Nie, P., & Yang, L. (2024) · Animals · doi:10.3390/ani14050707 · Cited by 12
“…important information in order to obtain superior results [33], and it has achieved great success in multiple visual tasks such as object detection, image classification, video understanding, semantic segmentation, image generation, multimodal tasks, 3D vision, and self-supervised learning [34].”
BackgroundMukta, M. S. H., Ahmad, J., Zaman, A., & Islam, S. (2024) · IEEE Access · doi:10.1109/access.2024.3373558 · Cited by 14
“Motivated by several studies [63], [64], we incorporate attention technique into our classification model, particularly for handling multimodal data.”
MethodWang, M., Ma, X., Wang, R., Kari, T., & Tang, Z. (2024) · Journal of Renewable and Sustainable Energy · doi:10.1063/5.0198444 · Cited by 12
Efficient YOLO-Based Deep Learning Model for Arabic Sign Language Recognition
Al-Ahmadi, S. A., Mohammad, F., & Dawsari, H. A. (2024) · Journal of Disability Research · doi:10.57197/jdr-2024-0051 · Cited by 12
Unveiling Hidden Insights in Gas Chromatography Data Analysis with Generative Adversarial Networks
Yoon, N., Jung, W., & Kim, H. (2024) · Chemosensors · doi:10.3390/chemosensors12070131 · Cited by 12
“The attention mechanism is a computational model that mimics the human cognitive system by selectively focusing on certain features or regions of input data, while filtering out irrelevant information [26].”
BackgroundA multi-head attention-like feature selection approach for tabular data
Pham, H., Tan, Y.-F., Singh, T., Pavlopoulos, V., & Patnayakuni, R. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112250 · Cited by 15
Compressed Deep Learning Models for Wearable Atrial Fibrillation Detection through Attention
Mäkynen, M., Ng, G. A., Li, X., Schlindwein, F. S., & Pearce, T. (2024) · Sensors · doi:10.3390/s24154787 · Cited by 15
“The attention mechanism can be categorized based on the type of information it focuses on, such as what, where, when, or which to pay attention to, with channel attention identified as the most computationally efficient method [36,38].”
BackgroundLiao, L., Wu, S., Song, C., & Fu, J. (2024) · Electronics · doi:10.3390/electronics13163149 · Cited by 12
Peng, Y., Liu, D., Deng, Y., Zhang, M., Yan, L., Luo, Y., & Yan, J. (2024) · Medical Physics · doi:10.1002/mp.17375 · Cited by 12
Bhavanam, S. R., Channappayya, S. S., Srijith, P. K., & Desai, S. (2024) · Astrophysics and Space Science · doi:10.1007/s10509-024-04357-9 · Cited by 12
“Attention CNN: As discussed in Section 1, attention mechanisms play a pivotal role in computer vision in enhancing the capability of models to focus on relevant information within an image (Guo et al, 2022).”
BackgroundDong, X., Tian, Y., Dai, L., Li, J., & Wan, L. (2024) · Aerospace · doi:10.3390/aerospace11090718 · Cited by 12
YOLO-Feder Fusionnet: A Novel Deep Learning Architecture for Drone Detection
Lenhard, T. R., Weinmann, A., Jäger, S., & Koch, T. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10647355 · Cited by 12
“Attention mechanisms frequently only concentrate on either spatial or channel-related feature relationships (cf. [27]).”
BackgroundMu, J., Su, Q., Wang, X., Liang, W., Xu, S., & Wan, K. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01562-1 · Cited by 13
Contactless Fingerprint Biometric Anti-Spoofing: An Unsupervised Deep Learning Approach
Adami, B., Hosseinzadehketilateh, M., & Karimian, N. (2024) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb62174.2024.10744434 · Cited by 12
“We enhance the classification performance of unsupervised model by incorporating attention mechanism into convolutional autoencoder [29] which has been used previously in many fields [30] [31].”
Cross-modal feature interaction network for heterogeneous change detection
Yang, Z., Wang, X., Lin, H., Li, M., & Lin, M. (2025) · Geo-spatial Information Science · doi:10.1080/10095020.2024.2446307 · Cited by 12
“Inspired by the workings of the human visual system, attention mechanisms are applied to networks to guide them in focusing on the parts of interest, thereby improving the model’s performance and interpretability (Guo et al. 2022).”
Sahragard, E., Farsi, H., & Mohamadzadeh, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0305561 · Cited by 15
Nan, S., Li, Z., Jin, S., Du, W., & Shen, W. (2025) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.4c03293 · Cited by 12
Wang, Y., Zhang, Y., Zhang, L., Wan, Y., Chen, Z., Xu, Y., Cao, R., Zhao, L., Yang, Y., & Yu, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113120 · Cited by 12
Liu, C., Shao, H., & Zhong, D. (2025) · IET Image Processing · doi:10.1049/ipr2.70029 · Cited by 12
A Spectral-Spatial Attention Network for Hyperspectral Unmixing
Tao, X., Koirala, B., Rasti, B., Plaza, A., & Scheunders, P. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3576479 · Cited by 12
“Very recently, inspired by the remarkable ability of the attention mechanism to capture long-range dependencies in images [43], [44], [45], several techniques have been proposed, which use attention to improve the extraction of spatial and spectral features from hyperspectral data for…”
Engineering artificial intelligence: framework, challenges, and future direction
Lee, J., Su, H., Ji, D.-Y., & Minami, T. (2025) · Machine Learning Engineering · doi:10.1088/3049-4761/adce0d · Cited by 28
“…[17, 51], predictive maintenance [12, 56], energy [68, 19], transportation [62, 70], healthcare [72, 53], design [64, 71], computer vision (CV) [13, 26], natural language processing (NLP) [43, 29], construction [66, 6], robotics and automation [49, 55], agriculture [52, 9], materials science [7,…”
SEAN: A Simple and Efficient Attention Network for Aircraft Detection in SAR Images
Han, P., Liao, D., Han, B., & Cheng, Z. (2022) · Remote Sensing · doi:10.3390/rs14184669 · Cited by 12
“The attention mechanism in computer vision can be regarded as the process of dynamically adjusting the weights based on the features of the input image [44].”
BackgroundAn Attention-Based Architecture for Hierarchical Classification With CNNs
Pizarro, I., Ñanculef, R., & Valle, C. (2023) · IEEE Access · doi:10.1109/access.2023.3263472 · Cited by 12
“Attention is a recent technique for improving deep learning models that have shown promising results in Natural Language Processing [19], Speech Recognition [38], and Computer Vision [39].”
MethodGuangyi, 吴. W., Zhuoqun, 袁. Y., & Liang, Y. (2023) · Acta Optica Sinica · doi:10.3788/aos230720 · Cited by 12
Optimization model based on attention mechanism for few-shot image classification
Liao, R., Zhai, J., & Zhang, F. (2024) · Machine Vision and Applications · doi:10.1007/s00138-023-01502-2 · Cited by 13
WT-YOLOM: An Improved Target Detection Model Based on YOLOv4 for Endogenous Impurity in Walnuts
Wang, D., Dai, D., Zheng, J., Li, L., Kang, H., & Zheng, X. (2023) · Agronomy · doi:10.3390/agronomy13061462 · Cited by 12
Unified Domain Adaptive Semantic Segmentation
Zhang, Z., Wu, G., Zhang, J., Zhu, X., Tao, D., & Chai, T. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3562999 · Cited by 14
“To improve inference efficiency, methods like attention-based spatial feature reconstruction [41] and weight-sharing subnetworks [42] have been proposed.”
MethodCategory-Guided Transformer for Semantic Segmentation of High-Resolution Remote Sensing Images
Ni, Y., Liu, J., Zhang, H., Chi, W., & Luan, J. (2025) · Remote Sensing · doi:10.3390/rs17173054 · Cited by 11
Self-citationMAD-Net: Multi-attention dense network for functional bone marrow segmentation
Qin, C., Zheng, B., Li, W., Chen, H., Zeng, J., Wu, C., Liang, S., Luo, J., Zhou, S., & Lin, X. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106428 · Cited by 11
Lim, W.-H., Bonab, M. B., & Chua, K. H. (2023) · IEEE Access · doi:10.1109/access.2023.3266737 · Cited by 13
“An attention mechanism performs a feature selection process by applying a learn-able weight to each feature of the model based on their importance [61].”
BackgroundJiang, Z., Shi, D., & Zhang, S. (2023) · The Visual Computer · doi:10.1007/s00371-023-03024-4 · Cited by 14
“Then, we introduce an SE attention block [34], which can emphasize important areas and suppress irrelevant background information based on the attention mechanism [35].”
Self-citationTowards efficient video-based action recognition: context-aware memory attention network
Koh, T. C., Yeo, C. K., Jing, X., & Sivadas, S. (2023) · SN Applied Sciences · doi:10.1007/s42452-023-05568-5 · Cited by 11
“The two common types of visual attention [54] are hard and soft attention.”
BackgroundAn Unsupervised Band Selection Method via Contrastive Learning for Hyperspectral Images
Li, X., Liu, Y., Hua, Z., & Chen, S. (2023) · Remote Sensing · doi:10.3390/rs15235495 · Cited by 11
“The attention mechanism, originally conceived for machine translation [29], has led to significant advances in areas such as natural language processing [30], speech [31], and computer vision [32–35].”
BackgroundData-free Knowledge Distillation for Fine-grained Visual Categorization
Shao, R., Zhang, W., Yin, J., & Wang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00146 · Cited by 11
“In terms of tasks, spatial attention is well-suited to dense prediction tasks such as semantic segmentation and object detection [14], while channel attention is a good choice for image classification.”
BackgroundRobust meter reading detection via differentiable binarization
Rao, Y., Guo, H., Liu, D., & Zeng, S. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05278-4 · Cited by 11
Incomplete multi-view clustering via self-attention networks and feature reconstruction
Zhang, Y., Jiang, L., Liu, D., & Liu, W. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05299-z · Cited by 11
NPA2Net: A Nested Path Aggregation Attention Network for Oriented SAR Ship Detection
Zhang, C. Z., Liu, P., Wang, H., & Jin, Y. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3392433 · Cited by 11
“…j ) = Node i,j ( P ( i − 1 , j ) , P ( i − 1 , j + 1)) j = 1 Node i,j ( P ( i − 1 , j ) , P ( i − 1 , j + 1) , P ( i, j − 1)) j (cid:5) = 1 (10) a) Channel attention module: The channel attention module adaptively recalibrates the weight of each channel and determines what to pay attention to [65].”
BackgroundGómez-Adorno, H., Bel-Enguix, G., Sierra, G., Barajas, J., & Alvarez-Giron, W. (2024) · Informatics · doi:10.3390/informatics11020024 · Cited by 17
“‘lbfgs’ is an optimizer in the family of quasi-Newton methods, ‘sgd’ refers to stochastic gradient descent, ‘adam’ refers to a stochastic gradient-based optimizer proposed by Kingma, Diederik [74].”
MethodPan, Y., Chen, Q., & Fang, X. (2024) · The Visual Computer · doi:10.1007/s00371-024-03543-8 · Cited by 11
“In a literature review [44] regarding the introduction of attention mechanisms, it was mentioned that dual attention methods extract more contextual information and achieve better results than single attention mechanisms.”
Hierarchical attention network for short-term runoff forecasting
Wang, H. E., Qin, H., Liu, G., Huang, S., Qu, Y., Qi, X., & Zhang, Y. (2024) · Journal of Hydrology · doi:10.1016/j.jhydrol.2024.131549 · Cited by 12
Anitha, V., Manimegalai, D., & Kalaiselvi, S. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112074 · Cited by 11
The hawk eye scan: Halyomorpha halys detection relying on aerial tele photos and neural networks
Palazzetti, L., Rangarajan, A. K., Dinca, A., Boom, B., Popescu, D., Offermans, P., & Pinotti, C. M. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109365 · Cited by 11
Raza, N., Habib, M. A., Ahmad, M., Abbas, Q., Aldajani, M. B., & Latif, M. A. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.055049 · Cited by 11
Enhanced YOLOv8 with attention mechanisms for accurate detection of colorectal polyps
Wang, S., Lin, S., Sun, F., & Li, X. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106942 · Cited by 11
Robust multi-stage progressive autoencoder for hyperspectral anomaly detection
Guo, Q., Cen, Y., Zhang, L., Zhang, Y., Hu, S., & Liu, X. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.104200 · Cited by 11
Self-citationYan, X., & Zhu, Y. (2024) · Asia Symposium on Image Processing (ASIP) · doi:10.1109/asip63198.2024.00013 · Cited by 12
Kia, M. (2024) · Iran Journal of Computer Science · doi:10.1007/s42044-024-00215-7 · Cited by 11
“These weights are typically learned during training and are dynamically adjusted in response to the input [52–54].”
Wang, Z., Huang, W., Qi, Z., & Yin, S. (2024) · Biomimetics · doi:10.3390/biomimetics9120784 · Cited by 11
A 3D Dual Encoder Mirror Difference ResU-Net for Multimodal Brain Tumor Segmentation
Xing, Q., Li, Z., Jing, Y., & Chen, X. (2024) · IEEE Access · doi:10.1109/access.2024.3522682 · Cited by 11
Sajid, M., Malik, K. R., Rehman, A. U., Malik, T. S., Alajmi, M., Khan, A. H., Haider, A., & Hussen, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-87376-y · Cited by 11
Zedda, L., Loddo, A., & Ruberto, C. D. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109704 · Cited by 13
Si, Q., & Han, S. I. (2025) · IEEE Access · doi:10.1109/access.2025.3544631 · Cited by 11
“The attention mechanism has proven highly effective in numerous visual tasks, including image classification, object detection, and semantic segmentation [44].”
An efficient feature aggregation network for small object detection in UAV aerial images
Liu, X., Zhang, G., & Zhou, B. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-06987-4 · Cited by 11
“Li et al. provided an extensive survey on attention mechanisms in computer vision, highlighting their efficacy in UAV scenarios [33].”
The Segmentation of Tunnel Faces in Underground Mines Based on the Optimized YOLOv5
Ma, C., Li, K., Pan, J., Zheng, J., Zhang, Q., & Qi, C. (2025) · Minerals · doi:10.3390/min15030255 · Cited by 11
Dynamic feature capturing in a fluid flow reduced-order model using attention-augmented autoencoders
Beiki, A., & Kamali, R. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110463 · Cited by 11
Real-Time Student Activity Detection and Incident Monitoring Using Artificial Intelligence
Kumar, N. S., SivaKrishna, K., Vamsi, V., Bhargavi, L., Nallagonda, A., & Goud, G. A. (2025) · International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT) · doi:10.1109/idciot64235.2025.10914692 · Cited by 11
Aldakhil, L. A., Alharbi, S. S., Aloraini, A., & Alhasson, H. F. (2025) · Diagnostics · doi:10.3390/diagnostics15060718 · Cited by 11
Parallel attention recursive generalization transformer for image super-resolution
Wang, J., Hao, Y., Bai, H., & Yan, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-92377-y · Cited by 11
Chen, J., Chen, G., Zhou, P., He, Y., Yue, L., Ding, M., & Hui, L. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104546 · Cited by 11
Classroom Behavior Detection Method Based on PLA-YOLO11n
Zhang, H., Zhou, G., He, W., & Deng, H. (2025) · Sensors · doi:10.3390/s25175386 · Cited by 11
“Although single-stage detectors excel in both speed and accuracy, they still struggle to reliably distinguish diverse behaviors in complex classroom environments and often underperform in recognizing small targets under occlusion or low-light conditions [32].”
A Unified Multiview Spectral Feature Learning Framework for Hyperspectral Image Classification
Li, X., Gu, Y., & Pižurica, A. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3213838 · Cited by 11
“The attention mechanism has been widely used in diverse deep learning models to boost the feature representations [46].”
MethodConvolution-Vision Transformer for Automatic Lung Sound Classification
Neto, J., Arrais, N., Vinuto, T. D. S., & Lucena, J. (2022) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi55357.2022.9991756 · Cited by 11
“CBAM is suitable to explore relationships of features to tell the network what and where to pay attention enhancing informative local regions [27].”
BackgroundMethodWaveNets: Wavelet Channel Attention Networks
Salman, H., Parks, C., Hong, S. Y., & Zhan, J. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020665 · Cited by 11
“[4], [5].”
Lightweight Remote-Sensing Image Super-Resolution via Re-Parameterized Feature Distillation Network
Zhang, T., Bian, C., Zhang, X., Chen, H., & Chen, S. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3258421 · Cited by 11
“The spatial and channel attention block has been proven effective in model enhancement from the spatial and channelwise dimensions, respectively [16].”
BackgroundA Semantic Transferred Priori for Hyperspectral Target Detection With Spatial–Spectral Association
Lei, J., Xu, S., Xie, W., Zhang, J., Li, Y., & Du, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3261302 · Cited by 11
“Attention mechanism can be categorized into four basic types [48]: channel attention [49], spatial attention [50], temporal attention [51], and branch attention [52], and two types of combined attention: channel–spatial attention [53] and spatial–temporal attention [54].”
Background2s-GATCN: Two-Stream Graph Attentional Convolutional Networks for Skeleton-Based Action Recognition
Zhou, S., Chen, R.-R., Jiang, X., & Pan, F. (2023) · Electronics · doi:10.3390/electronics12071711 · Cited by 13
Adaptive Multispectral Encoding Network for Image Demoiréing
Dai, Q. G., Cheng, X., Zhang, L., & Sun, L.-K. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3280518 · Cited by 11
“According to Guo et al. [25], four primary divisions of visual attention exist: branch attention, channel attention, spatial attention, and temporal attention.”
MethodXie, Y., Liu, Q., Ji, C., Sun, Y., Zhang, S., Hua, M., Liu, X., Pan, S., Hu, W., Ma, Y., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-35556-z · Cited by 12
Self-citationLi, R., Chen, T., Liu, Y., & Jiang, H. (2023) · International Journal of Remote Sensing · doi:10.1080/01431161.2023.2255353 · Cited by 13
“In Tables 2 and 3, we present a comparative analysis with state-of-the-art methods such as U-Net, Deeplabv3+ (Chen et al. 2018), D-LinkNet, Swin Transformer, PVT, TransUNeT, and RoadFormer, which are based on CNNs, ViTs, and hybrid CNN-Transformer architectures.”
MethodInfluentialSong, B., Wang, J., Wang, X., Zeng, T., & Li, D. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105197 · Cited by 11
Recognition of Converter Steelmaking State Based on Convolutional Recurrent Neural Networks
Huang, C., Dai, Z., Sun, Y., Wang, Z., Liu, W., Yang, S., & Li, J. (2024) · Metallurgical and Materials Transactions B · doi:10.1007/s11663-024-03071-9 · Cited by 11
Enhancing additive manufacturing with computer vision: a comprehensive review
Yu, H., Li, W., Li, D., Wang, L., & Wang, Y. (2024) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-024-13689-3 · Cited by 12
“…sensing technology with strong intuition, low cost, and high efficiency, CV tasks mainly included classification, retrieval, super-resolution, segmentation and object detection of images, temporal, and spatial detection of videos, and reconstruction of three-dimensional models and scenes [12, 13].”
Bridging knowledge distillation gap for few-sample unsupervised semantic segmentation
Li, P., Chen, J., & Tang, C. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120714 · Cited by 11
YOLO-DFAN: Effective High-Altitude Safety Belt Detection Network
Yan, W., Wang, X., & Tan, S. (2022) · Future Internet · doi:10.3390/fi14120349 · Cited by 11
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
