The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 501–600 of 1,001
Image Super-Resolution via Lightweight Attention-Directed Feature Aggregation Network
Li, W., Li, K., Tang, J., & Liang, Y. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3546076 · Cited by 16
“More importantly, the atention mechanism can be seen as a dynamic selection process implemented by adaptively weighting features according to the importance of the input information [12].”
BackgroundAttention Mechanisms for Physiological Signal Deep Learning: Which Attention Should We Take?
Park, S.-A., Lee, H., Jung, C.-W., & Yang, H.-L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16431-6_58 · Cited by 16
The Positive Effect of Attention Module in Few-Shot Learning for Plant Disease Recognition
Lin, H., Tse, R. T., Tang, S.-K., Qiang, Z., & Pau, G. (2022) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai55851.2022.9904046 · Cited by 18
“Attention mechanism plays important role in human perception to selectively focus on salient parts in order to capture visual structure better [25].”
BackgroundZhan, M., Shi, X., Liu, F., & Hu, R. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2022.103258 · Cited by 16
An Ensemble Learning Approach with Multi-depth Attention Mechanism for Road Damage Detection
Wang, S., Tang, Y., Liao, X., He, J., Feng, H., Jiao, H., Su, X. B., & Yuan, Q. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10021018 · Cited by 16
“The human visual system was introduced into deep learning technology, which is called attention mechanisms [22].”
BackgroundLiu, Z., Chen, Q., Wei, H., Lv, M., & Chen, L. (2023) · Measurement · doi:10.1016/j.measurement.2023.113097 · Cited by 16
Self-citationBright-field to fluorescence microscopy image translation for cell nuclei health quantification
Wang, R., Butt, D., Cross, S., Verkade, P., & Achim, A. (2023) · Biological Imaging · doi:10.1017/s2633903x23000120 · Cited by 17
“This approach can be transferred to spatial analysis in image processing((44)).”
BackgroundZhao, S., Li, G., Zhou, M., & Li, M. (2023) · Cluster Computing · doi:10.1007/s10586-023-04079-7 · Cited by 16
A Multi-Agent Reinforcement Learning Method for Cooperative Secondary Voltage Control of Microgrids
Wang, T., Ma, S., Tang, Z., Xiang, T., Mu, C., & Jin, Y. (2023) · Energies · doi:10.3390/en16155653 · Cited by 18
Chen, S., Jin, T., Xia, Y., & Li, X. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3313593 · Cited by 16
“Its major objective is to imitate how the human visual system focuses on salient parts in complex scenes by dynamically adjusting the weights based on the input image features [30].”
BackgroundLi, L.-H., & Tanone, R. (2023) · Journal of ICT Research and Applications · doi:10.5614/itbj.ict.res.appl.2023.17.2.3 · Cited by 16
“In model development, various architectures have been introduced, such as CNN to Vision Transformer for computer vision tasks [15,16].”
MethodLi, G., Xiao, H., Li, W., & Luo, Y. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1392-6 · Cited by 16
Yang, Z., Li, K., & Huang, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108110 · Cited by 17
Foroutan, P., & Lahmiri, S. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100552 · Cited by 20
Learning data distribution of three-dimensional ocean sound speed fields via diffusion models
Li, S., Cheng, L., Li, J., Wang, Z., & Li, J. (2024) · The Journal of the Acoustical Society of America · doi:10.1121/10.0026026 · Cited by 18
Multi-Branch Network for Color Image Denoising Using Dilated Convolution and Attention Mechanisms
Duong, M.-T., Thi, B.-T. N., Lee, S., & Hong, M.-C. (2024) · Sensors · doi:10.3390/s24113608 · Cited by 16
“To overcome this problem, we leverage the attention mechanism [40], which is widely used for diverse image restoration tasks [41,42], into the middle of the encoder and decoder.”
MethodZhang, Z., Peng, W., Alsalman, O., Ma, L., Huang, J., & Zhu, C. G. (2024) · Optics Express · doi:10.1364/oe.514466 · Cited by 16
UAV Sensing-Based Litchi Segmentation Using Modified Mask-RCNN for Precision Agriculture
Deka, B., & Chakraborty, D. (2024) · IEEE Transactions on AgriFood Electronics · doi:10.1109/tafe.2024.3420028 · Cited by 20
“…multilevel cross-scale feature fusion [12] and embedding attention mechanisms into the network: channel attention, spatial attention, temporal attention, and branch attention [22] that can accurately detect and count objects in real time, while minimizing the associated computational burden.”
BackgroundYang, W., & Qiu, X. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1395796 · Cited by 16
“As a result, attention mechanisms have demonstrated substantial potential in computer vision (Guo et al., 2022); among them, the channel attention mechanism is employed to enhance the representation capability of each channel within CNNs.”
Attention‐enhanced multiscale feature fusion network for pancreas and tumor segmentation
Dong, K., Hu, P., Zhu, Y., Tian, Y., Li, X., Zhou, T., Bai, X., Liang, T., & Li, J. (2024) · Medical Physics · doi:10.1002/mp.17385 · Cited by 18
A survey on cell nuclei instance segmentation and classification: Leveraging context and attention
Nunes, J. D., Montezuma, D., Oliveira, D., Pereira, T., & Cardoso, J. S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103360 · Cited by 19
Cross-Modal Feature Fusion for Field Weed Mapping Using RGB and Near-Infrared Imagery
Fan, X., Ge, C., Yang, X., & Wang, W. (2024) · Agriculture · doi:10.3390/agriculture14122331 · Cited by 16
“In contrast, Transformer-based architectures [27] can effectively leverage attention mechanisms to capture global semantic information, establishing long-range dependencies between objects.”
Bai, M., Di, X., Yu, L., Ding, J. J., & Lin, H. (2025) · Remote Sensing · doi:10.3390/rs17020255 · Cited by 16
Zhang, Q., Lian, X., Qin, J., Duan, J., & Zhou, Y. (2025) · Measurement · doi:10.1016/j.measurement.2025.116991 · Cited by 16
Li, C., Liu, C., Ju, W., Zhong, Y., & Li, Y. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00240-w · Cited by 16
“Attention mechanisms are used to fuse multimodal data, and different weights are assigned to each modality, effectively integrating educational information, highlighting the most critical information for prediction tasks, and improving the predictive performance of teaching quality [31, 32].”
An Efficient Deep Learning Model for Prostate Cancer Diagnosis
Alici-Karaca, D., & Akay, B. (2024) · IEEE Access · doi:10.1109/access.2024.3472209 · Cited by 15
“In this formulation, while f ( x ) refers to the attention generation process that draws attention to the distinctive regions, the expression A ( f ( x ) , x ) refers to the processing of x input together with the generated attention [49].”
BackgroundLu, X., Poon, J., & Khushi, M. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06462-w · Cited by 15
A survey of deep learning methods on cell instance segmentation
Wang, C., Lee, W.-T., & Su, T.-S. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11119-3 · Cited by 21
Eren, B. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111725 · Cited by 15
Feng, F., Zhang, Y., Zhang, J., & Liu, B. (2023) · Remote Sensing · doi:10.3390/rs15020304 · Cited by 15
Chen, H., Wang, Y., Peng, J., Zhang, R., & Jiang, P. (2023) · Applied Sciences · doi:10.3390/app13095589 · Cited by 15
Self-citationZhang, Z., Zhang, Q., Hu, X., Zhang, M., & Zhu, D. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.05.026 · Cited by 15
Multi-modal adaptive gated mechanism for visual question answering
Xu, Y., Zhang, L., & Shen, X. (2023) · PLoS ONE · doi:10.1371/journal.pone.0287557 · Cited by 18
“The principle of the attention mechanism [15, 16] is to give different weights to different inputs by imitating the way humans observe objects so that the feature selection can be focused on and the model’s prediction accuracy can be improved.”
Shi, L., Zhang, J., Yang, B., & Gao, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3306911 · Cited by 15
“tion can be regarded as a dynamic branch selection mechanism: in a multi-branch structure, which branches are worth paying attention to [27].”
BackgroundHybrid attention mechanism of feature fusion for medical image segmentation
Tong, S., Zuo, Z., Liu, Z., Sun, D., & Zhou, T. (2023) · IET Image Processing · doi:10.1049/ipr2.12934 · Cited by 15
Self-citationSelf-supervised deep monocular visual odometry and depth estimation with observation variation
Zhao, W., Wang, Y., Wang, Z., Li, R., Xiao, P., Wang, J., & Guo, R. (2023) · Displays · doi:10.1016/j.displa.2023.102553 · Cited by 15
Guo, W., Zhang, Y., Peng, C., Geng, X., Jiang, M., Zhang, L., Sui, Q., & Zhang, F. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3323276 · Cited by 15
“Lately, the attention mechanism has been applied in many fields to enhance the interpretability of the model [13], [14], [15], exploring the potential relationship between the input and the output.”
MethodTwo-Stage Domain Adaptation for Infrared Ship Target Segmentation
Zhang, T., Shen, H., Rehman, S. U., Liu, Z., Li, Y., & Rehman, O. U. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3325298 · Cited by 15
Self-citationZhu, Y., Fan, L., Li, Q., & Jing, C. (2023) · Remote Sensing · doi:10.3390/rs15215243 · Cited by 15
“It can adaptively recalibrate the weight assigned to elements of interest via the input image features [59–61].”
SMSS: Stateful Model Serving in Metaverse With Serverless Computing and GPU Sharing
Cai, Z., Chen, Z., Ma, R., & Guan, H. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3345401 · Cited by 14
“Machine learning techniques, such as deep learning and neural networks, have led to breakthroughs in various fields, including computer vision [8], [9], natural language processing [10], [11], and recommendation systems [12].”
BackgroundStereo Image Restoration via Attention-Guided Correspondence Learning
Zhang, S., Yu, W., Jiang, F., Nie, L., Yao, H., Huang, Q., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357709 · Cited by 16
“…of multi-dimensional features by convolution, the attention mechanism biases the allocation of resources to the most challenging regions and disregards irrelevant regions, which effectively establishes long-range dependencies between features and thus benefits many computer vision tasks [38], [39].”
BackgroundSelf-citationGAEFS: Self-supervised Graph Auto-encoder enhanced Feature Selection
Tan, J., Gui, N., & Qiu, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111523 · Cited by 17
Automatic breast mass segmentation in ultrasound images with U-Net and resolution enhancement blocks
Rahmani, A. A., Shirazi, A. A. B., & Behnam, H. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106270 · Cited by 15
ScorePillar: A Real-Time Small Object Detection Method Based on Pillar Scoring of Lidar Measurement
Cao, Z., Wang, T., Sun, P. B., Cao, F., Shao, S., & Wang, S. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3378251 · Cited by 16
“…the first 32-Dof K j , and M s is the last dimension of K j . f ( · ) is computed as follows : b) Mutual-scoring unit based on attention mechanism: An attention mechanism is a method that allows an algorithm to focus on the most important parts of the input and ignore other interfering parts [39].”
MethodZhao, S., Zhong, R. Y., Jiang, Y., Besklubova, S., Tao, J., & Yin, L. (2024) · Computers & Industrial Engineering · doi:10.1016/j.cie.2024.110220 · Cited by 18
Zhang, S., Qi, X., Duan, J., Yuan, X., Zhang, H., Feng, W., Guo, T., & He, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3401474 · Cited by 15
“This approach facilitates the extraction and concentration of pivotal features, and further amalgamates local dynamic information to bolster the model’s predictive precision [70].”
BackgroundSelf-citationHDA-pose: a real-time 2D human pose estimation method based on modified YOLOv8
Dong, C., Tang, Y., & Zhang, L. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03274-2 · Cited by 15
“The Spatial Attention [44, 45] enables the model to dynamically select relevant feature regions that are highly correlated with the human body poses, while ignoring less important regions of the image.”
MDANet: Modality-Aware Domain Alignment Network for Visible-Infrared Person Re-Identification
Cheng, X., Yu, H., Cheng, K. H. M., Yu, Z., & Zhao, G. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521822 · Cited by 16
“Attention Mechanism: Recently, attention mechanisms have been widely utilized in computer vision [23], [24], [25], [26], [27], [37], [38], [39], [40].”
Advances in artificial intelligence: a review for the creative industries
Anantrasirichai, N., Zhang, F., & Bull, D. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11494-w · Cited by 62
FOOL: Addressing the Downlink Bottleneck in Satellite Computing With Neural Feature Compression
Furutanpey, A., Zhang, Q., Raith, P., Pfandzelter, T., Wang, S., & Dustdar, S. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3544516 · Cited by 30
“The advantage of 3D layers over concatenating channels and applying 2D convolutional operations (e.g., with channel attention [54]) is that it considerably reduces width.”
Yu, Y., Jiang, T., Li, Y., Guan, H., Li, D., Chen, L., Yu, C., Gao, L., Gao, S., & Li, J. (2022) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.08.011 · Cited by 15
Point cloud completion via structured feature maps using a feedback network
Su, Z., Huang, H., Ma, C., Huang, H. F., & Hu, R. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0276-6 · Cited by 15
Multi-Attention-Based Semantic Segmentation Network for Land Cover Remote Sensing Images
Jia, J., Song, J., Kong, Q., Yang, H., Teng, Y., & Song, X. (2023) · Electronics · doi:10.3390/electronics12061347 · Cited by 15
Wang, X., Kang, M., Chen, Y., Jiang, W., Wang, M., Weise, T., Tan, M., Xu, L., Li, X., Zou, L., et al. (2023) · Remote Sensing · doi:10.3390/rs15081980 · Cited by 15
Deep learning-assisted Raman spectroscopy for automated identification of specific minerals
Dong, W., Qin, M., Wu, S., Hu, L., Rao, C., & Du, Z. (2025) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2025.125843 · Cited by 14
Transformer-Based Transfer Learning for Battery State-of-Health Estimation
Giuliano, A., Wu, Y., Yawney, J., & Gadsden, S. A. (2025) · Energies · doi:10.3390/en18205439 · Cited by 14
Sivaanpu, A., Punithakumar, K., Zheng, R., Nguyen, K. T., Noga, M., Ta, D., Lou, E. H. M., & Le, L. H. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.109150 · Cited by 14
AS-Net: An attention-aware downsampling network for point clouds oriented to classification tasks
Yang, Y., Wang, A., Bu, D., Feng, Z., & Liang, J. (2022) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103639 · Cited by 14
Hong, R., Xiao, B., He, Y., Liu, J., Liu, P., & Song, Z. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.107689 · Cited by 14
Self-citationTian, F., Hu, G., Yu, S., Wang, R., Song, Z., Yan, Y., Huang, H., Wang, Q., Wang, Z., & Yu, Z. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.108000 · Cited by 14
Cervical cell nuclei segmentation based on GC-UNet
Zhang, E., Xie, R., Bian, Y., Wang, J., Tao, P., Zhang, H., & Jiang, S. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e17647 · Cited by 14
“The utilization of context information to facilitate better cell map segmentation has been demonstrated in various research studies [35].”
BackgroundLiang, Z., Cui, G., Xiong, M., Li, X., Jin, X., & Lin, T. (2023) · Agronomy · doi:10.3390/agronomy13081988 · Cited by 14
Yang, Z., Wu, Y., Li, M., Hu, X., & Li, Z. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103511 · Cited by 14
Zhang, X., Cao, X., Zhang, H., Shen, Y., Yuan, X., Cui, Z., & Lu, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3324145 · Cited by 14
“…learning to focus on important feature information, attention mechanisms (AEs) has been applied to CNN models and shown its effectiveness to extract key information and improve model performance, which include several types: spatial domain, channel domain, temporal domain and mixed domain [36].”
MethodLi, X., Wang, L., Miao, H. Y., & Zhang, S. (2023) · Insects · doi:10.3390/insects14110839 · Cited by 14
“( attention mechanism: CBAM is a lightweight attention mechanism that combines channel and space, consisting of a Channel Attention Module (CAM) and a Spatial Attention Module (SAM) [39].”
BackgroundSelf-citationYOLO‐UOD: An underwater small object detector via improved efficient layer aggregation network
Chen, W., Zhuang, T., Zhang, Y., Mei, T., & Tang, X. (2024) · IET Image Processing · doi:10.1049/ipr2.13112 · Cited by 14
Zhao, L., Qiu, S., & Chen, Y. (2024) · Sensors · doi:10.3390/s24103104 · Cited by 14
“Traditional attention mechanisms are categorized into channel attention, spatial attention, branch attention, and hybrid attention [33].”
BackgroundConditional generative adversarial networks for image-based sunlight analysis of residential blocks
Hou, D. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114295 · Cited by 15
Artificial intelligence model for tumoral clinical decision support systems
Iglesias, G., Talavera, E., Troya, J., Díaz-Álvarez, A., & García-Remesal, M. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108228 · Cited by 15
“One of the techniques in which AI has recently achieved better results is in Computer Vision (CV) [23].”
Deep learning-based detection and condition classification of bridge elastomeric bearings
Dong, L., Zhang, S., Huang, H., Zhang, L., & Hu, Y. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105680 · Cited by 14
Self-citationEpiTESTER: Testing Autonomous Vehicles With Epigenetic Algorithm and Attention Mechanism
Lu, C., Ali, S., & Yue, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3449429 · Cited by 14
“The attention mechanism is crucial in modern machine learning applications, such as natural language processing [25] and computer vision [26].”
BackgroundChen, D., Gao, Z., Liu, J., Song, T., Li, L., & Tian, L. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106926 · Cited by 14
Self-citationXu, Z., Li, H., & Long, B. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125779 · Cited by 14
Wu, P., Zhou, H., Yu, Y., Miao, Z., Pan, Q., Zhang, X., & Gao, J. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3524584 · Cited by 15
“The attention mechanism plays a crucial role in enabling models to selectively focus on relevant regions containing valuable information amidst background noise [26].”
Qi, J., Li, W., Bai, Z., Gao, H., Tang, X., & Zhou, Y. (2025) · Journal of Environmental Management · doi:10.1016/j.jenvman.2025.124584 · Cited by 14
Li, X.-J., Yang, J., & Zhang, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19818-2_31 · Cited by 14
“The survey papers [25,37,67,42] have comprehensive discussion on the attention mechanism.”
BackgroundSBMYv3 : Improved MobYOLOv3 a BAM attention‐based approach for obscene image and video detection
Samal, S., Zhang, Y., Gadekallu, T. R., Nayak, R., & Balabantaray, B. K. (2023) · Expert Systems · doi:10.1111/exsy.13230 · Cited by 14
“Attention module has a significant impact on the training experience (Guo et al., 2022).”
BackgroundNiu, T., Xie, Z., Zhang, J., Tang, L., Li, B., & Wang, H. (2023) · Computers in Industry · doi:10.1016/j.compind.2023.103978 · Cited by 14
TriagedMSA: Triaging Sentimental Disagreement in Multimodal Sentiment Analysis
Luo, Y., Liu, W., Sun, Q., Li, S., Li, J., Wu, R., & Tang, X. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3524789 · Cited by 15
“It has now become a crucial component of neural networks and is extensively employed in multimodal learning for capturing intra-and inter-modal relations [24], [25].”
Ayanwale, M. A., Adelana, O. P., Bamiro, N. B., Olatunbosun, S. O., Idowu, K. O., & Adewale, K. A. (2025) · F1000Research · doi:10.12688/f1000research.161637.1 · Cited by 15
LSODNet: A Lightweight and Efficient Detector for Small Object Detection in Remote Sensing Images
Jin, K., Du, W., Tang, M., Liang, W., Li, K., & Pathan, A. K. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3612261 · Cited by 14
“Attention mechanisms have emerged as a simple yet powerful tool for improving neural network representations, finding extensive use across various architectures [40].”
YOLO-DP: A detection model of fifteen common rice diseases and pests
Zhou, T., & Wei, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-19310-1 · Cited by 16
“Current research incorporates Triplet Attention into the YOLOv8n Backbone, enriching the model’s information exchange with auxiliary keys and bolstering its ability to discern intricate structures 26 .”
Skew Class-Balanced Re-Weighting for Unbiased Scene Graph Generation
Kang, H., & Yoo, C. D. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010018 · Cited by 15
“Currently, computer vision [1–6] has been prevalent in various fields: material [7], chemical [8,9], and medical science [10].”
Depthwise Temporal Non-Local Network for Faster and Better Dynamic Hand Gesture Authentication
Song, W., & Kang, W. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3256708 · Cited by 13
“The current mainstream attention mechanisms [46] are broadly categorized into channel attention [47], [48], [49], [50], [51], [52], spatial/spatiotemporal attention [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], temporal attention [42], [65], as well as…”
BackgroundCNN Attention Enhanced ViT Network for Occluded Person Re-Identification
Wang, J., Li, P., Zhao, R., Zhou, R., & Han, Y. (2023) · Applied Sciences · doi:10.3390/app13063707 · Cited by 13
Fault Diagnosis for Marine Two-Stroke Diesel Engine Based on CEEMDAN-Swin Transformer Algorithm
Cui, D., & Hu, Y. (2023) · Journal of Failure Analysis and Prevention · doi:10.1007/s11668-023-01684-x · Cited by 13
Obscenity detection transformer for detecting inappropriate contents from videos
Rautela, K., Sharma, D., Kumar, V., & Kumar, D. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16078-2 · Cited by 13
TSVR-Net: An End-to-End Ground-Penetrating Radar Images Registration and Location Network
Bi, B., Shen, L., Zhang, P., Huang, X., Xin, Q., & Jin, T. (2023) · Remote Sensing · doi:10.3390/rs15133428 · Cited by 13
WiDual: User Identified Gesture Recognition Using Commercial WiFi
Dai, M., Cao, C., Liu, T., Su, M., Li, Y., & Li, J. (2023) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid57682.2023.00068 · Cited by 13
“Each feature map has two dimensions, embedding positional information into channel attention will help the network to focus on large important regions at little computational cost [16].”
BackgroundInfluentialCross-Attention-Guided Feature Alignment Network for Road Crack Detection
Xu, C., Zhang, Q., Mei, L., Chang, X., Ye, Z., Wang, J., Ye, L., & Yang, W. (2023) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi12090382 · Cited by 13
Li, Y., Wu, Z., Shen, J., & Zhang, Q. (2023) · Optics Express · doi:10.1364/oe.506343 · Cited by 13
STCA-SNN: self-attention-based temporal-channel joint attention for spiking neural networks
Wu, X., Song, Y., Zhou, Y., Jiang, Y., Bai, Y., Li, X., & Yang, X. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1261543 · Cited by 14
“A popular research direction is to present attention as an auxiliary module that can be easily integrated with existing architectures to boost the representation power of the basic model (Hu et al., 2018; Woo et al., 2018; Guo et al., 2022; Li et al., 2022).”
BackgroundVisual complexity of urban streetscapes: human vs computer vision
Florio, P., Leduc, T., Sutter, Y., & Brémond, R. (2023) · Machine Vision and Applications · doi:10.1007/s00138-023-01484-1 · Cited by 13
“Other studies try to implement attention mechanisms in computer vision, imitating the high-level extraction of key information [29, 30].”
BackgroundRethinking the role of attention mechanism: a causality perspective
Wang, C., & Zhou, Y. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05279-3 · Cited by 13
SVSDet: A Fine-Grained Recognition Method for Ship Target Using Satellite Video
Liu, S., Bu, X., Xu, M., Sheng, H., Zeng, Z., & Yasir, M. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3359252 · Cited by 13
“The attention mechanism can be considered a dynamic weight adjustment process for input image features that can effectively find significant areas in complex scenes by imitating human vision [59].”
BackgroundSelf-Attentive Contrastive Learning for Conditioned Periocular and Face Biometrics
Ng, T.-S., Chai, J. C. L., Low, C.-Y., & Teoh, A. B. J. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3361216 · Cited by 13
“We direct the readers to a comprehensive survey of these endeavors [27].”
BackgroundDu, W., Fan, Z., Yan, Y., Yu, R., & Liu, J. (2024) · Remote Sensing · doi:10.3390/rs16091574 · Cited by 13
Self-citationA systematic study of DNN based speech enhancement in reverberant and reverberant-noisy environments
Wang, H., Pandey, A., & Wang, D. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101677 · Cited by 13
Feng, G., Li, Z., Zhang, J., & Wang, M. (2024) · Sensors · doi:10.3390/s24144728 · Cited by 13
“The spatial attention mechanism, a pivotal element within convolutional neural networks, functions as an adaptive process that selectively focuses on key spatial areas, thus addressing the question of “where to focus” [21].”
BackgroundAttention-Based Multi-Modal Multi-View Fusion Approach for Driver Facial Expression Recognition
Jianrong, C., Dey, S., Wang, L., Bi, N., & Liu, P. (2024) · IEEE Access · doi:10.1109/access.2024.3462352 · Cited by 13
“Attention can be considered a dynamic selection process in a computer vision system, which is achieved by weighting features according to their importance [22].”
Background
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
