The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 401–500 of 1,001
Local-Global Decompositions for Conditional Microstructure Generation
Robertson, A. E., Kelly, C., Buzzy, M. O., & Kalidindi, S. R. (2023) · Acta Materialia · doi:10.1016/j.actamat.2023.118966 · Cited by 21
Liu, Z., Cheng, Y., Tan, T., & Shinichi, T. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110394 · Cited by 20
Self-citationLi, X., Wang, X., Sun, S., Wang, Y., Li, S., & Li, D. (2023) · Fire Technology · doi:10.1007/s10694-023-01427-2 · Cited by 20
Li, C., Xu, R., Lv, Y., Zhao, Y., & Jing, W. (2023) · Remote Sensing · doi:10.3390/rs15163975 · Cited by 22
RA-UNet: an improved network model for image denoising
Liu, W., Li, Y., & Huang, D. (2023) · The Visual Computer · doi:10.1007/s00371-023-03084-6 · Cited by 20
High-Accuracy and Fine-Granularity Human Activity Recognition Method Based on Body RFID Skeleton
Zheng, H., Chen, Y., Li, Y., Wang, Z., Chen, J., & Xiao-lin, Z. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3340752 · Cited by 20
“The Adam algorithm [37] is employed to optimize the parameters of the proposed HARs using recurrent neural networks and graph convolutional networks.”
MethodZhao, C., Guo, D., Shao, C., Zhao, K., Sun, M., & Shuai, H. (2024) · IEEE Access · doi:10.1109/access.2024.3382245 · Cited by 20
Pham, X. L., Luu, H. M., van Walsum, T., Mai, H. S., Klein, S., Le, N. H., & Duc, T. C. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103212 · Cited by 20
DeepPCC: Learned Lossy Point Cloud Compression
Zhang, J., Liu, G., Zhang, J., Ding, D., & Ma, Z. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3467192 · Cited by 20
“Inspired by the impressive performance of self-attention in image/point cloud classification and segmentation [22], [23], [24], in this work, we propose to apply sparse convolution [25] and local self-attention [26] to strictly limit the computations to positively occupied voxels (POV) only in close…”
Luo, X., Liu, D., Kong, H., Huai, S., Chen, H., Xiong, G., & Liu, W. (2024) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3701728 · Cited by 27
Liang, Y., & Xu, Z. (2023) · Engineering Construction & Architectural Management · doi:10.1108/ecam-07-2023-0705 · Cited by 19
Mapping urban villages based on point-of-interest data and a deep learning approach
Li, T., Feng, Q., Niu, B., Chen, B., Yan, F., Gong, J., & Liu, J. (2024) · Cities · doi:10.1016/j.cities.2024.105549 · Cited by 23
Self-citationPigmented skin disease classification via deep learning with an attention mechanism
Chen, J., Jiang, Q., Ai, Z., Wei, Q., Xu, S., Hao, B., Lü, Y.-P., Huang, X., & Chen, L. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112571 · Cited by 24
Liu, F., Geng, H., Shang, L., Si, C., & Shen, S. (2025) · Plant Methods · doi:10.1186/s13007-025-01357-w · Cited by 19
Oyemakinde, T. T., Kulwa, F., Peng, X., Liu, Y., Cao, J., Deng, X., Wang, M., Li, G., Samuel, O. W., Fang, P., et al. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3552811 · Cited by 19
“This mechanism was introduced to deep learning to enable it to effectively and efficiently capture salient regions of the feature map [43].”
Liu, J., Wen, B., Xiao, J., & Sun, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130207 · Cited by 24
Self-citationAgent-based multimodal information extraction for nanomaterials
Odobesku, R., Romanova, K., Mirzaeva, S., Zagorulko, O., Sim, R., Khakimullin, R. R., Razlivina, J., Dmitrenko, A., & Vinogradov, V. V. (2025) · npj Computational Materials · doi:10.1038/s41524-025-01674-7 · Cited by 26
ULO: An Underwater Light-Weight Object Detector for Edge Computing
Wang, L., Ye, X., Wang, S., & Li, P. (2022) · Machines · doi:10.3390/machines10080629 · Cited by 19
Jiang, M., Zhang, X., Sun, Y., Feng, W., Gan, Q., & Ruan, Y. (2022) · GIScience & Remote Sensing · doi:10.1080/15481603.2022.2142626 · Cited by 21
“The attention mechanism (Guo et al. 2022a) was successfully applied to image vision and can extract meaningful information from complicated features and improve the accuracy of remote sensing image change detection using deep learning methods (Liu et al. 2022; Guo et al. 2022b).”
MethodLiu, Y., Liu, X., Zhang, Y., & Li, S. (2022) · Entropy · doi:10.3390/e25010071 · Cited by 20
“Recently, the attention model has become an important concept and has been wildly used in NLP [10] and Computer Vision [11].”
MethodSaihood, A., Karshenas, H., & Naghsh‐Nilchi, A. R. (2023) · IEEE Access · doi:10.1109/access.2023.3243104 · Cited by 21
“Attention mechanism [44] has been proposed to overcome these limitations.”
BackgroundA Self-attention Guided Multi-scale Gradient GAN for Diversified X-ray Image Synthesis
Saad, M. M., Rehmani, M. H., & O’Reilly, R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-26438-2_2 · Cited by 19
Yao, Z., Zhang, W., Song, P., Hu, Y., & Liu, J. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad095 · Cited by 19
Self-citationZhang, P., Wang, Z., Rao, Y., Zheng, J. J., Zhang, N., Wang, D., Zhu, J., Fang, Y., & Gao, X. (2023) · Forests · doi:10.3390/f14030588 · Cited by 19
Cui, B., Zhao, Y., Yang, M., Huang, L., & Lu, Y. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3281823 · Cited by 19
“The attention mechanism is a method of shifting attention to the most important regions of an image while ignoring irrelevant regions [28].”
BackgroundMethodModification and Evaluation of Attention-Based Deep Neural Network for Structural Crack Detection
Yuan, H., Jin, T., & Ye, X. (2023) · Sensors · doi:10.3390/s23146295 · Cited by 19
“Attention mechanisms in deep learning [36,37] are very similar to human visual attention mechanisms, which select more important information for the current target and remove redundant information.”
Meng, D.-D., Li, S., Sheng, B., Wu, H., Tian, S., Ma, W., Wang, G.-P., & Yan, X. (2023) · The Visual Computer · doi:10.1007/s00371-023-02965-0 · Cited by 19
Jin, Z., Qian, J., Kong, Z., & Pan, C. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109981 · Cited by 20
Wang, H., Lin, Z., Zhou, J., Xing, L., & Zeng, P. (2023) · IEEE International Conference on Contemporary Computing and Communications (InC4) · doi:10.1109/inc457730.2023.10263077 · Cited by 19
“These various attention mechanisms provide valuable tools for modulating attention and enhancing performance in diverse perceptual tasks.[18].”
BackgroundTeaDiseaseNet: multi-scale self-attentive tea disease detection
Sun, Y., Wu, F. Q., Guo, H., Li, R., Yao, J., & Shen, J. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1257212 · Cited by 20
Bi-Source Reconstruction-Based Classification Network for Face Forgery Video Detection
Zhang, D., Fu, C., Lu, D., Li, J., & Zhang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3330390 · Cited by 22
“Feature aggregation and improvement [44] are designed to combine and choose salient feature details in deep neural networks, whose effectiveness has been discussed in many works, including image classification, object detection, and semantic segmentation.”
BackgroundExploring deep learning approaches for video captioning: A comprehensive review
Yousif, A. J., & Al-Jammas, M. H. (2023) · e-Prime - Advances in Electrical Engineering Electronics and Energy · doi:10.1016/j.prime.2023.100372 · Cited by 25
Li, Q., Huang, T., Liu, J., & Tan, L. (2024) · Journal of Manufacturing Processes · doi:10.1016/j.jmapro.2024.01.082 · Cited by 19
Self-citationWang, W. J., Wang, X., Huang, Y. F., Zhao, Y., Fang, X., Cong, Y., Tang, Z., Chen, L.-Z., Zhong, J., Li, R., et al. (2024) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-024-05209-9 · Cited by 19
A deep multi-branch attention model for histopathological breast cancer image classification
Ding, R., Zhou, X., Tan, D., Su, Y., Jiang, C., Yu, G., & Zheng, C.-H. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01398-z · Cited by 20
“There are two main types of attentional mechanisms [39]: 1) Channel Attention, such as SeNet [7], which focuses on allowing the network to learn what to pay attention; 2) Spatial Attention, such as self-attention [24], which focuses on allowing the network to learn where to pay attention.”
A cross-stage features fusion network for building extraction from remote sensing images
Zuo, X., Shao, Z., Wang, J., Huang, X., & Wang, Y. (2024) · Geo-spatial Information Science · doi:10.1080/10095020.2024.2307922 · Cited by 23
“However, the VHR remote sensing images also make the homogeneous and heterogeneous effects more obvious and introduce more noise, for example, houses and roads are very similar, making it difficult to extract the fine building results (Guo et al. 2022).”
Ali, H., Wang, M., & Xie, J. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10289-x · Cited by 19
Li, W., Xia, H., Zhang, J., Wang, Y., Jia, Y. M., & He, Y. (2024) · Remote Sensing · doi:10.3390/rs16162908 · Cited by 23
“For image classification, attention mechanisms are mainly classified into the following categories depending on the data area of action: channel attention, spatial attention, and channel and spatial attention [50].”
BackgroundTransformers-based architectures for stroke segmentation: a review
Zafari-Ghadim, Y., Rashed, E. A., Mohamed, A. M. S., & Mabrok, M. A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10900-5 · Cited by 25
“This architectural choice is based on empirical evidence suggesting that combining spatial and channel attention enhances the model’s capacity to capture more contextual features (Guo et al. 2022).”
Dong, X., Zhang, L., Xu, C. X., Miao, Q., Yao, J., Liu, F., Liu, H., Lu, Y., Kang, R., & Song, B. (2024) · Ecological Indicators · doi:10.1016/j.ecolind.2024.112764 · Cited by 19
Mou, T., & Wang, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-85139-3 · Cited by 19
“However, the traditional text mining method has many limitations when dealing with online comments of tourist attractions, such as the inability to accurately identify the emotional tendency in comments and the poor effect of topic classification 14,15 .”
Predicting significant wave height in the South China Sea using the SAC-ConvLSTM model
Hou, B., Fu, H., Li, X., Song, T., & Zhang, Z. (2024) · Frontiers in Marine Science · doi:10.3389/fmars.2024.1424714 · Cited by 19
Wen, X., Maimaiti, M., Liu, Q., Yu, F., Gao, H., Li, G., & Chen, J. (2024) · Plants · doi:10.3390/plants13162334 · Cited by 18
“…into the field of computer vision, imitating the ability of the human visual system to focus on salient regions in complex scenes and categorize those regions according to various approaches, such as channel attention, spatial attention, temporal attention, branch attention, and so on [14].”
BackgroundEfficient crop segmentation net and novel weed detection method
Kong, X., Liu, T., Chen, X., Jin, X., Li, A., & Yu, J. (2024) · European Journal of Agronomy · doi:10.1016/j.eja.2024.127367 · Cited by 18
Materials, processes, devices and applications of magnetoresistive random access memory
Yang, M., Cui, Y., Chen, J., & Luo, J. (2024) · International Journal of Extreme Manufacturing · doi:10.1088/2631-7990/ad87cb · Cited by 19
“…single-layer perceptron in the 1950s [216], the multi-layer perceptron [217–220], the convolutional neural network (CNN) [221], the recurrent neural network (RNN) [222], and the spiking neural network (SNN) [223–225], generative adversarial network (GAN) [226], and self-attention mechanism [227].”
Krishnamoorthy, U., & Balasubramani, S. (2024) · Advanced Theory and Simulations · doi:10.1002/adts.202400479 · Cited by 18
Zolfaghari, M., & Sajedi, H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309674 · Cited by 18
“This selection focuses more on particular regions of space, specific features of an object, or entire of them in a real or imagined scene [60].”
A deep learning-based probabilistic approach to flash flood warnings in mountainous catchments
Zhao, Y., Wu, X., Zhang, W., Lan, P., Qin, G., Li, X., & Li, H. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.132677 · Cited by 21
Real-Time Cross-Domain Gesture and User Identification via COTS WiFi
Cao, C., Ding, Y., Dai, M., Gong, W., & Zhao, X. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3532295 · Cited by 18
“Incorporating positional information into the channel attention mechanism allows the network to focus on significant regions with minimal computational expense [17].”
InfluentialUsing UAV-based multispectral images and CGS-YOLO algorithm to distinguish maize seeding from weed
Tang, B., Zhou, J., Zhao, C., Pan, Y., Lu, Y., Liu, C., Ma, K., Sun, X., Zhang, R., & Gu, X. (2025) · Artificial Intelligence in Agriculture · doi:10.1016/j.aiia.2025.02.007 · Cited by 18
Li, C., Han, X., Yao, C., Guo, Y., Li, Z., Jiang, L., Liu, W., Huang, H., Fu, H., & Ban, X. (2025) · Acta Materialia · doi:10.1016/j.actamat.2025.120962 · Cited by 18
Aerodynamic performance prediction of 3D aircraft based on point cloud deep learning
Wang, S., Zhang, X., Tie, Y., Xia, W., & Jiang, Y. (2025) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwaf051 · Cited by 19
Zhang, Z.-Q., Huang, T., Qian, B., & Hu, R. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111466 · Cited by 19
Huang, L. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/3934381 · Cited by 18
“*e student representation vector P is used as the query vector to calculate the weight of attention, which is used to calculate the preference degree of students for different characteristics of employment units [34, 35].”
MethodWang, J., Zhang, C., Yan, T., Yang, J., Lu, X., Lu, G., & Huang, B. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00955-8 · Cited by 20
“Attention mechanisms are often embodied as plug-and-play attention modules that can refine convolutional outputs within a block and enable the whole network to learn more informative features [50].”
BackgroundReview of Data-Driven Approaches for Wind Turbine Blade Icing Detection
Cai, C., Guo, J., Song, X., Zhang, Y., Wu, J., Tang, S., Jia, Y. M., Xing, Z., & Li, Q. (2023) · Sustainability · doi:10.3390/su15021617 · Cited by 18
“Through performing continuous iterative updates, the model learns to give more weight to the robust features and less weight to the weak features [66].”
Li, W., Liu, W., Shao, S., Huang, S., & Song, A. (2023) · IEEE Transactions on Games · doi:10.1109/tg.2023.3263013 · Cited by 19
“Attention mechanism, which can automatically select the significant information from data, has multidisciplinary and multifield applications [9], [37], [38], [39], [40], [41].”
BackgroundWang, Y., Yu, X., Guo, X., Wang, X., Wei, Y., & Zeng, S. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103856 · Cited by 18
Interaction in Transformer for Change Detection in VHR Remote Sensing Images
Chen, Z., Song, Y., Ma, Y., Li, G., Wang, R., & Hu, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3324025 · Cited by 18
“In the visual system, the attention adaptively weights the network based on the input region’s importance, performing differential attention [35].”
BackgroundYang, B., Guo, Z., Wang, L., He, J., Xia, B., & Vakily, S. (2023) · Remote Sensing · doi:10.3390/rs15204971 · Cited by 19
Constrained tandem neural network assisted inverse design of metasurfaces for microwave absorption
He, X., Cui, X., & Chan, C. T. (2023) · Optics Express · doi:10.1364/oe.506936 · Cited by 18
Channel Features and API Frequency-Based Transformer Model for Malware Identification
Qian, L., & Lin, C. (2024) · Sensors · doi:10.3390/s24020580 · Cited by 26
“However, the self-attention mechanism has some drawbacks when encoding information at the current position, and it tends to concentrate attention excessively on its own position [34].”
BackgroundA Deep Learning Approach for Forecasting Thunderstorm Gusts in the Beijing-Tianjin-Hebei Region
Liu, Y., Yang, L., Chen, M., Song, L., Han, L., & Xu, J. (2024) · Advances in Atmospheric Sciences · doi:10.1007/s00376-023-3255-7 · Cited by 20
Wang, H., Du, W., Xu, G., Sun, Y., & Shen, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-61396-6 · Cited by 20
“Attention mechanisms are divided into channel, space, branch and temporal attention mechanisms 27 .”
BackgroundLearning Sea Surface Height Interpolation From Multi‐Variate Simulated Satellite Observations
Archambault, T., Filoche, A., Charantonis, A., Béréziat, D., & Thiria, S. (2024) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2023ms004047 · Cited by 19
“It is widely used in many computer vision tasks (Guo et al., 2021) and can be transposed to geoscience applications such as (Che et al., 2022).”
BackgroundGao, M., Li, S., Chen, X., & Wang, Y. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109145 · Cited by 18
Lin, J., Zhang, H., & Shang, H. (2024) · Bioengineering · doi:10.3390/bioengineering11101021 · Cited by 18
“…mechanism, detailed in Algorithm 2, can be executed in parallel on multiple attention heads, which effectively improves the model’s expressiveness in different feature dimensions [48,49], gives it a more global perspective, and is especially suitable for processing complex image data.”
BackgroundA nuclide identification method of γ spectrum and model building based on the transformer
Li, F., Luo, C., Wen, Y., Lv, S., Feng, C., Zeng, G.-Q., Jiang, J.-F., & Li, B. (2024) · Nuclear Science and Techniques · doi:10.1007/s41365-024-01564-5 · Cited by 18
“To make full use of brain resources, it will not process all information at the highest granularity, but will focus on the parts of interest and apply brain resources to (1) them [40].”
SelectE: Multi-scale adaptive selection network for knowledge graph representation learning
Zu, L., Lin, L., Fu, S., Guo, F., & Wu, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111554 · Cited by 18
Wang, Z., Guo, L., Zhao, S., Zhang, S., Zhao, X., Fang, J., Wang, G., Lu, H., Yu, J., & Tian, Q. C. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3523112 · Cited by 17
“The attention mechanism [48] originates from human cognition, and emphasizes significant information contents.”
Self-citationZhao, Z., Lee, C. K. M., Tsang, Y. P., & Xu, X. (2025) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2025.105031 · Cited by 17
A lightweight CNN model for UAV-based image classification
Deng, X., Shi, M., Khan, B., Choo, Y. H., Ghaffar, F., & Lim, C. P. (2025) · Soft Computing · doi:10.1007/s00500-025-10512-3 · Cited by 17
“It has been widely exploited to improve DL performance in computer vision, yielding impressive results in a variety of visual and related tasks, such as image classification, object detection, image segmentation, and self-supervised learning (Guo et al. 2022; Niu et al. 2021).”
Sample less meta-learning fault diagnosis based on ordered time–frequency features
Wang, C., Wang, N., & Deng, L. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110881 · Cited by 17
Low-light multimodal object detection: A survey
Li, G., Wang, Y., He, B., Pang, T., & Gao, M. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100804 · Cited by 17
Danish, S., Piran, M. J., Khan, S. U., Khan, M. A., Dang, L. M., Zweiri, Y., Song, H., & Moon, H. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11415-3 · Cited by 19
“However, it has some limitations like high complexity, huge data for training, and reduced interpretability, risk of overfitting (Guo et al. 2022 , 2023; Saleh et al. 2024).”
Face Mask-Wearing Detection Model Based on Loss Function and Attention Mechanism
Wang, Z., Sun, W., Zhu, Q., & Shi, P. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/2452291 · Cited by 17
“Attention mechanisms have achieved great success in various computer vision tasks [23]. purpose of introducing an attention mechanism is to select key information from many redundant pieces of information to submit the detection ability of the network.”
BackgroundRINet: Relative Importance-Aware Network for Fixation Prediction
Song, Y., Liu, Z., Li, G., Zeng, D., Zhang, T., Xu, L., & Wang, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3249481 · Cited by 17
“The attention mechanism imitates this inherent characteristic of human vision by adaptively weighting features according to the importance of the input [47].”
BackgroundSelf-citationVDDNet: An Object Detection Model for Measuring the Vibration Displacement of Rotating Structure
Huo, L., Mao, J., Wang, S., & San, H. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3268083 · Cited by 17
Mining Semantic Information With Dual Relation Graph Network for Multi-Label Image Classification
Zhou, W., Jiang, W., Chen, D., Hu, H., & Su, T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3277279 · Cited by 17
“In general, each channel map in the features focuses on a spe-cific part of image [45], [46].”
BackgroundRyu, J. S., Lee, S., Chu, Y., Ahn, M., Park, Y. J., & Yang, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286916 · Cited by 17
Wang, S., Yu, X., Li, Y., Wang, S., & Meng, C. (2023) · Earth Science Informatics · doi:10.1007/s12145-023-01103-7 · Cited by 17
“The specific calculation is as follows (Guo et al. 2022): 1) Use dot product operation to calculate the similarity between each input vector at each time step and the output of the BiLSTM layer.”
MethodHuo, T., Deng, L., Zhang, B., Gong, J., Hu, B., Zhao, R., & Liu, Z. (2023) · Measurement Science and Technology · doi:10.1088/1361-6501/ad006a · Cited by 17
Self-citationWu, X., Li, L., Zhu, H., Zhou, G., Li, L., Su, F., He, S., Wang, Y., & Long, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122363 · Cited by 17
Erep, T. R. B. A., & Chaâri, L. (2023) · Italian National Conference on Sensors · doi:10.3390/s24010209 · Cited by 17
“Attention mechanisms have proven to be effective in computer vision, allowing models to focus on relevant parts of the input by assigning different weights to different regions [46].”
BackgroundShin, M., Seo, M., Yoo, S., & Yoon, K. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3389708 · Cited by 17
“Transformers have been shown to outperform traditional CNN-based models in SR in several studies [41], [44], [45], [46].”
BackgroundFlood Water Depth Prediction with Convolutional Temporal Attention Networks
Chaudhary, P., Leitão, J. P., Konrad Schindler, B., & Wegner, J. D. (2024) · Water · doi:10.3390/w16091286 · Cited by 17
Where is my attention? An explainable AI exploration in water detection from SAR imagery
Chen, L., Cai, X., Li, Z., Jin, X., & Ai, J. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103878 · Cited by 17
Chinese Water Demand Forecast Based on iTransformer Model
Tian, Z.-W., & Qian, R.-L. (2024) · IEEE Access · doi:10.1109/access.2024.3446663 · Cited by 18
“The Transformer architecture, celebrated for its achievements in natural language processing and computer vision [36], [37], [38], [39], [40], has emerged as a fundamental model in accordance with the scaling principle [37], [38], [39], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50],…”
BackgroundUSSC-YOLO: Enhanced Multi-Scale Road Crack Object Detection Algorithm for UAV Image
Zhang, Y., Lu, Y., Huo, Z., Li, J., Sun, Y., & Huang, H. (2024) · Sensors · doi:10.3390/s24175586 · Cited by 17
Lee, Y.-S., Patil, M. P., Kim, J. G., Choi, S. S., Seo, Y. B., & Kim, G. (2024) · Agriculture · doi:10.3390/agriculture14091472 · Cited by 19
Real-time detection of dead fish for unmanned aquaculture by yolov8-based UAV
Zhang, H., Tian, Z., Liu, L., Liang, H., Feng, J., & Zeng, L. (2024) · Aquaculture · doi:10.1016/j.aquaculture.2024.741551 · Cited by 17
Probabilistic Attention Map: A Probabilistic Attention Mechanism for Convolutional Neural Networks
Liu, Y., & Tian, J. (2024) · Sensors · doi:10.3390/s24248187 · Cited by 19
Wind power forecasting using a GRU attention model for efficient energy management systems
Boucetta, L. N., Amrane, Y., & Arezki, S. (2024) · Electrical Engineering · doi:10.1007/s00202-024-02590-7 · Cited by 16
Pan, X., Jiao, C., Yang, B., Zhu, H., & Wu, J. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107332 · Cited by 17
Deformation prediction model for concrete dams considering the effect of solar radiation
Liu, M., Qi, Y., & Su, H. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103252 · Cited by 16
GrainNet: efficient detection and counting of wheat grains based on an improved YOLOv7 modeling
Wang, X., Li, C., Zhao, C., Jiao, Y., Xiang, H., Wu, X., & Chai, H. (2025) · Plant Methods · doi:10.1186/s13007-025-01363-y · Cited by 16
Thabet, R. M., Shedeed, H. A., Al-Berry, M. N., & Khattab, D. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11143-8 · Cited by 17
“Techniques such as attention mechanisms (Guo et al. 2022), saliency maps (Boyd et al. 2022), or simpler models can help bridge this gap between model complexity and interpretability.”
Zhu, Y.-K., Wan, H., Ye, Y., & Todd, M. D. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.113005 · Cited by 16
Multiscale Gaussian Attention Mechanism for Tiny-Object Detection in Remote Sensing Images
Shi, S., Fang, Q., Xu, X., & Dong, D. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3591260 · Cited by 19
“Recent computer vision studies classify object detection attention mechanisms into three categories [31]: channel-based approaches (SKNet [19], SENet [17], ECA-Net [18]), spatial-based approaches (DCN [20]), and hybrid channel–spatial frameworks like CBAM [21].”
Xu, J., Yang, J., Huang, Y., Por, L. Y., Chen, X., & Zhao, C. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129518 · Cited by 16
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
