The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 801–900 of 1,001
DAAR: Dual attention cooperative adaptive pruning rate by data-driven for filter pruning
Lian, S., Yang, Z., & Pei, J. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06332-5 · Cited by 8
Alkayal, S., Almisbahi, H., Baowidan, S. A., & Alkayal, E. (2025) · Atmosphere · doi:10.3390/atmos16020211 · Cited by 8
“These metrics were used to compare the predictive performance of the models across the three cities.”
UAS-based MT-YOLO model for detecting missed tassels in hybrid maize detasseling
Qi, J., Ding, C., Zhang, R., Xie, Y., Li, L., Zhang, W., & Chen, L. (2025) · Plant Methods · doi:10.1186/s13007-025-01341-4 · Cited by 9
“To mitigate this challenge, integrating an attention mechanism that dynamically enhances the weight of relevant tassel features within complex environments can significantly improve feature extraction and detection accuracy [33].”
GLAAM and GLAAI: Pioneering attention models for robust automated cataract detection
Kumar, D., Verma, C., & Illés, Z. (2025) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2025.100182 · Cited by 8
An explainable and comprehensive BI-RADS assisted diagnosis pipeline for mammograms
Li, P., Zhong, J., Chen, H., Hong, J., Li, H., Li, X., & Shi, P. (2025) · Physica Medica · doi:10.1016/j.ejmp.2025.104949 · Cited by 10
Zhang, L., Sun, S., Zhao, H., Li, Z., & Li, D. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103125 · Cited by 8
SiamCTCA: Cross-Temporal Correlation Aggregation Siamese Network for UAV Tracking
Wang, Q., Liu, F., Zhang, B., Liu, J., Xu, F., & Wang, Y. (2025) · Drones · doi:10.3390/drones9040294 · Cited by 8
“The core idea of the attention mechanism is to simulate the selective attention mechanism in the human visual or cognitive process so that the model can dynamically focus on the most relevant information [25,26].”
Self-citationA novel transformers-based external attention framework for breast cancer diagnosis
Xi, R., Ahmed, A., Zeng, X., & Hou, M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108065 · Cited by 8
Graphically Residual Attentive Network for tackling aerial image occlusion
Pradhan, P. K., Purkayastha, K., Sharma, A. L., Baruah, U., Sen, B., & Ghosal, P. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110429 · Cited by 8
Multi-attention-powered learning genetic algorithm for real-world 3D wind farm layout optimization
Yang, J., Song, Y., Ding, W., Tang, J., Lei, Z., & Gao, S. (2025) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2025.102018 · Cited by 8
Pei, S., Fan, T., Shen, J., Zhang, X., & Du, X. (2025) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2025.214057 · Cited by 8
MTA-Net: Multi-scale triplet attention-aware network for multiclass skin lesion classification
Gajera, H. K., Nayak, D. R., & Zaveri, M. A. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110729 · Cited by 10
Pan, A., Xiang, Q., Tang, H., Wu, K., & Cheng, L. (2025) · Materials Characterization · doi:10.1016/j.matchar.2025.115486 · Cited by 8
Liang, L., Shi, H., Wang, Z., Wang, S., Li, C.-H., & Diao, M. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1652478 · Cited by 10
“The AT Module dynamically weights and aggregates input information through the coordinated interaction of three vectors: Query (Q), Key (K), and Value (V) (Guo et al., 2022; Tian et al., 2022).”
Shen, X., Dong, Y., Yu, H., Hao, N., Ping, J., Wang, W. D., Song, M., Wang, Y., Liu, C., Yu, H., et al. (2025) · Measurement · doi:10.1016/j.measurement.2025.119151 · Cited by 8
Wang, R., Chen, Y., Wu, H., Liu, J., Wang, M., & Duan, J. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.134366 · Cited by 8
Self-citationAdaptive Smooth Adversarial Learning for Cross-Domain Hyperspectral Image Classification
Liu, R., Zhou, M., Yang, J., & Wang, J. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3620393 · Cited by 8
“CNN attention mechanisms can be broadly categorized into six classes [52]: channel attention, spatial attention, temporal attention, branching attention, channel–spatial attention, and spatiotemporal attention.”
Advancements in 3-D object detection: A comprehensive review
Wu, W.-J., Appiah, I., & Hu, R. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00213-0 · Cited by 9
DynRouteNet: Lightweight Facial Shadow Removal With Mask-Guided Dynamic Routing
Zeng, Y., Chen, X., Li, Z., Zhang, X., Wang, M., Dong, Y., Zhang, X., Pun, C.-M., & Tsang, K.-F. (2025) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2025.3632210 · Cited by 8
“This DynamicBlock module dynamically selects between a Channel Attention Block (CAB) [38] and a Separable Convolution (SepConv2d) [39] based on input features and mask characteristics, ensuring targeted refinement of shadowed regions.”
Hanafy, A. E., Hessane, A., & Farhaoui, Y. (2025) · Technologies · doi:10.3390/technologies13120596 · Cited by 8
“…attention mechanisms were pioneered in natural language processing [10], attention modules have made their way into computer vision specifically due to their ability to dynamically emphasize the most informative parts of an input image and consequently aid classification and interpretation [11].”
CS-YOLO:A small object detection model based on YOLO for UAV aerial photography
Fan, R., Jiao, R., Nan, W., Meng, H., Jiang, A., Yang, X., Zhao, Z., Dang, J., Wang, Z., Tian, Y., et al. (2025) · Signal Processing Image Communication · doi:10.1016/j.image.2025.117460 · Cited by 10
Deep Learning-Based Layer Identification of 2D Nanomaterials
Zhang, Y., Zhang, H., Zhou, S., Liu, G., & Zhu, J. (2022) · Coatings · doi:10.3390/coatings12101551 · Cited by 8
“…nanomaterials, where data acquisition and training are difficult, data can be expanded by generative adversarial networks [68], data augmentation [69], and semi ‐ supervised [70] or weakly supervised [71] training. of data tagging tools according to the type and characteristics of 2D nanomaterials.”
BackgroundStripe Sensitive Convolution for Omnidirectional Image Dehazing
Zhao, D., Li, J., Li, H., & Xu, L. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3233900 · Cited by 8
“This is because the attention mechanism introduces special components, enabling the network to focus on ‘where’ or/and ‘what’ to pay attention [16].”
BackgroundCrop Mapping Based on Sentinel-2 Images Using Semantic Segmentation Model of Attention Mechanism
Gao, M., Lu, T., & Wang, L. (2023) · Sensors · doi:10.3390/s23157008 · Cited by 8
“For this reason, the attention mechanism was introduced into computer vision, the principle of which is to imitate the human cognitive process of selectively focusing on important information and ignoring secondary information [28].”
BackgroundStrategies for inserting attention in computer vision
Wu, J., Liu, X., & Dong, J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17373-8 · Cited by 8
“The soft attention mechanism directly uses the expectation of the context vector to guide the computer vision model for learning weights [12].”
Ndiaye, Y., Lim, K. H., & Blessing, L. T. M. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1170348 · Cited by 11
“…engineering education, AIET-based approaches have the potential to provide automatic and non-intrusive assessment (Meza et al., 2017; Ahrens, 2020; Chen, 2021), higher accuracy (Hijazi et al., 2021), complex dynamic scenes such as video-based data (Guo et al., 2022), and a less consuming process.”
Key Point Estimate Network for Rail-Track Detection
Yang, S., Wang, Z., Yu, G., & Liu, W. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3327996 · Cited by 8
“…scrutinize the entirety of an image rather than focusing specifically on the railway region, leading to inaccurate outcomes in rail-track detection [16]; Thirdly, at turnouts, rail-tracks intersect one another, resulting in parts of the rail-track system remaining undetected; Fourthly, the…”
BackgroundSTEAM: Spatial Trajectory Enhanced Attention Mechanism for Abnormal UAV Trajectory Detection
Yoon, N., Lee, D., Kim, K., Yoo, T., Joo, H., & Kim, H. (2023) · Applied Sciences · doi:10.3390/app14010248 · Cited by 8
Wang, Y., Han, X., Wei, X., & Luo, J. (2024) · Mathematics · doi:10.3390/math12010153 · Cited by 8
Hernández-Nava, G., Salazar-Colores, S., Cabal‐Yépez, E., & Ramos-Arreguín, J. M. (2024) · Sensors · doi:10.3390/s24030716 · Cited by 9
Municipal and Urban Renewal Development Index System: A Data-Driven Digital Analysis Framework
Wang, X., Li, X., Wu, T., He, S., Zhang, Y., Ling, X., Chen, B., Bian, L., Shi, X., Zhang, R., et al. (2024) · Remote Sensing · doi:10.3390/rs16030456 · Cited by 11
“The attention mechanism [54] is applied to help improve the performance of the DeeplabV3+ model.”
MethodApplication of Split Coordinate Channel Attention Embedding U2Net in Salient Object Detection
Wu, Y., & Wu, Y. (2024) · Algorithms · doi:10.3390/a17030109 · Cited by 8
Zhang, L., Li, Z., Guang, J., Xie, Y., Shi, Z., Gu, H., & Zheng, Y. (2024) · Atmosphere · doi:10.3390/atmos15030384 · Cited by 8
Zhang, C. N., Xie, L., Lü, X., Mao, R., Xu, L., & Xu, X. (2024) · Molecules · doi:10.3390/molecules29071499 · Cited by 8
“The attention mechanism is a widely employed technique in deep learning to enhance model performance by dynamically assigning weights to different segments of input data [29–31 , 35– 37 ].”
MethodMultiscale Global Attention for Abnormal Geological Hazard Segmentation
Cai, W., Huang, W., Tian, L., Huang, C., & Yan, J. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3383233 · Cited by 8
“The self-attention mechanism is vitally important in improving efficient and accurate deep neural networks [22].”
BackgroundG2EMOT: Guided Embedding Enhancement for Multiple Object Tracking in Complex Scenes
Ma, J., Wu, F., Li, C., Tang, C., Zhang, J., Xu, Z., Li, M., & Liu, D. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3449952 · Cited by 8
“Following the successful application of the attention mechanism in image recognition and video analysis [36], [37], several MOT methods [38], [39], [40] adopted a streamlined Transformer encoder-decoder architecture to simultaneously address object detection, data association, and trajectory…”
MethodEnhancing Microgrid Performance Prediction with Attention-Based Deep Learning Models
Maddineni, V. K., Koganti, N. B., & Damacharla, P. (2024) · International Conference on Information Technology, Computer, and Electrical Engineering · doi:10.1109/icitacee62763.2024.10762767 · Cited by 8
“This dictates the importance of each occurrence of the query[14].”
Image forgery classification and localization through vision transformers
Pawar, D., Gowda, R., & Chandra, K. (2025) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-025-00358-8 · Cited by 8
Attention-Guided Disentangled Feature Aggregation for Video Object Detection
Muralidhara, S., Hashmi, K. A., Pagani, A., Liwicki, M., Stricker, D., & Afzal, M. Z. (2022) · Sensors · doi:10.3390/s22218583 · Cited by 8
“Attention can be regarded as a dynamic mechanism that assesses the importance of the features and adapts the weights accordingly [32].”
BackgroundMethodZhai, X., Xu, J., Chen, N., Shen, J., Li, Y., Wang, Y., Chu, X., & Zhu, Y. (2022) · Journal of Educational Computing Research · doi:10.1177/07356331221115663 · Cited by 8
“Furthermore, to focus on the image ’ s task-related area effectively, channel attention enables the neural network to determine automatically which channel is important or unimportant and then assign the appropriate weights (Guo et al., 2022).”
BackgroundNovel Dynamic Feature Fusion Stragegy for Detection of Small Underwater Marine Object
Chen, J., Er, M. J., Zhang, Y., Gao, W., & Wu, J. (2022) · Intelligent Autonomous Systems · doi:10.1109/icoias56028.2022.9931237 · Cited by 8
“1) Spatial-aware Attention: Spatial-aware attention grants focus on special locations on the feature map selectively, allowing extraction of precise features in the space [25].”
BackgroundZou, Y., Wang, R., Du, M., Wang, X., & Xu, D. (2023) · The Journal of Physical Chemistry B · doi:10.1021/acs.jpcb.2c07592 · Cited by 9
Wei, Z., Yan, Q., Lu, X., Zheng, Y., Sun, S., & Lin, J. (2023) · Mathematics · doi:10.3390/math11040847 · Cited by 8
Soylu, U., & Oelze, M. L. (2023) · IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control · doi:10.1109/tuffc.2023.3245988 · Cited by 8
“self-attention mechanisms, [24] were invented initially for computer vision tasks, where the data is abundant and they apply attention by altering network architecture and hence model complexity to improve classification accuracy.”
Lei, L., Li, H., & Yang, H. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119014 · Cited by 8
Uncertainty-guided joint attention and contextual relation network for person re-identification
Wang, D., Chen, Y., Wang, W., Tie, Z., Fang, X., & Ke, W. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103822 · Cited by 8
Transformer Model for Fault Detection from Brazilian Pre-salt Seismic Data
Bomfim, L. D. S., Cunha, O. R., Kuroda, M. C., Vidal, A. C., & Pedrini, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45389-2_1 · Cited by 8
DGSN: Learning how to segment pedestrians from other datasets for occluded person re-identification
Liu, Y., Wang, Z., Zhang, W., & Li, Z. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104844 · Cited by 8
Microstructure recognition of steels by machine learning based on visual attention mechanism
Chen, X., Cheng, L., Hu, C., Zhang, Y.-P., & Wu, K. (2023) · Journal of Iron and Steel Research International · doi:10.1007/s42243-023-01031-2 · Cited by 8
A mixed‐attention‐based multi‐scale autoencoder algorithm for fabric defect detection
Zhang, H., Wu, Y., Lu, S., Yao, L., & Li, P. (2023) · Coloration Technology · doi:10.1111/cote.12725 · Cited by 7
Zero-DCE xT: A Computational Approach to Addressing Low-Light Image Enhancement Challenges
Ahmadi, C., Chen, J.-L., & Hsiao, Y.-Y. (2024) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2024.3392600 · Cited by 7
Attention-SA: Exploiting Model-Approximated Data Semantics for Adversarial Attack
Li, Q., Hu, Q., Fan, H., Lin, C., Shen, C. Q., & Wu, L. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3409945 · Cited by 7
“To achieve this, we leverage the attention mechanism [66], [67], [68] attached as a sub-module in targeted model, to distill the relative importance information of learned attributes.”
MethodLiu, S., Zheng, C., Wang, J., Lu, Y., Yao, J., Zou, Z., Yin, Y., & He, T. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1368885 · Cited by 7
Few-Shot Steel Defect Detection Based on a Fine-Tuned Network with Serial Multi-Scale Attention
Liu, X., Jiao, L., Peng, Y., An, K., Wang, D., Lu, W., & Han, J. (2024) · Applied Sciences · doi:10.3390/app14135823 · Cited by 7
Learning Piecewise Planar Representation for RGB Guided Depth Super-Resolution
Xu, R., Yao, M., Guan, Y., & Xiong, Z. (2024) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2024.3439990 · Cited by 7
“Different from the commonly used spatial-wise attention [61], [62], the planar attention projects the features to planar representations for filtering the modality-inconsistent information and identifying the geometry information.”
MethodEvaluation of cucumber seed germination vigor under salt stress environment based on improved YOLOv8
Fang, Z., Fan, Q., Tian, L., Jiang, H., Wang, C., Fu, X., Xiao-Zhong, L., Li, M., Zhang, S., Zhang, Y., et al. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1447346 · Cited by 7
“Attention Mechanism (AM) (Niu et al., 2021; Guo et al., 2022) is an approach inspired by studies of human vision.”
BackgroundSelf-citationMulti-stream feature aggregation network with multi-scale supervision for single image dehazing
Wu, J., Tao, H., Xiao, K., Chu, J., & Leng, L. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109486 · Cited by 8
Dynamical system simulation with attention and recurrent neural networks
Fañanás-Anaya, J., Nicolás, G. L., & Sagüés, C. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10732-y · Cited by 7
“Originally introduced in the context of natural language processing (NLP) [48], attention mechanisms, notably popularized by their incorporation into the transformer architecture, have since then found applications in diverse fields, ranging from computer vision to time series forecasting [49–51].”
Li, R., Hong, W., Wu, R., Wang, Y., Wu, X., Shi, Z., Xu, Y., Han, Z., & Lv, C. (2024) · Plants · doi:10.3390/plants13243462 · Cited by 7
“In image recognition tasks, the attention mechanism effectively helps models focus on important areas within an image, thereby significantly enhancing performance in classification and detection [40,41].”
MarsSeg: Mars Surface Semantic Segmentation With Multilevel Extractor and Connector
Li, J., Chen, K., Tian, G., Li, L., & Shi, Z. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3526630 · Cited by 7
“The attention mechanism, inspired by the human vision system, empowers the model to selectively concentrate on pertinent information within images [44].”
Exploring prototype-guided strategy for domain decomposition in physics-informed neural network
Wang, Y., Peng, Y., Hu, Z., & Li, Y. (2025) · Nonlinear Dynamics · doi:10.1007/s11071-025-10871-4 · Cited by 7
Wu, M., Gui, N., Yang, X., Tu, J. Y., & Jiang, S. (2025) · International Communications in Heat and Mass Transfer · doi:10.1016/j.icheatmasstransfer.2025.108763 · Cited by 7
Cao, X., Gao, H., Qin, T. G., Zhu, M., Zhang, P., & Xu, P. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1549544 · Cited by 7
“Channel attention assigns different weights to different channels, which can be regarded as a process of object selection (33).”
A miniaturized electronic nose system based on sensor array for C2H4 concentration prediction
Liu, Z., Liu, Z., Wu, J., Feng, P., Chu, J., & Peng, X. (2025) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2025.107189 · Cited by 7
Self-citationLiew, C. Y., Lim, J. M.-Y., Tan, C. P., & Mohar, R. M. M. B. T. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110814 · Cited by 7
Wei, Q., Wei, W., Ye, X., & Yao, B. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3561309 · Cited by 7
“Attention mechanisms [24], [25], [26], [27], [28] have been proven effective in diverse vision tasks, such as classification, object detection, and semantic segmentation, and hence are employed in stereo matching.”
MSPE-Fusion: A multimodal 3D object detection method with multi-sensor perception enhanced fusion
Yu, S., Wang, X., Ci, Y., Li, Y., Zhou, M., Dong, X., & Cai, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130486 · Cited by 9
Fu, J., Xu, Z., Yue, Q., Lin, J., Zhang, N., Zhao, Y., & Gu, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-02865-4 · Cited by 7
Guthrie, J. D., Shankarnarayan, S. A., & Charlebois, D. A. (2026) · Scientific Reports · doi:10.1038/s41598-026-48015-2 · Cited by 9
“However, vision transformers require more data than CNNs to reach comparable performance 25 .”
Optical image processing and applications empowered by vision-language models
Xiao, J., Sun, Z., An, H., Zhao, H., Qiu, M., & Li, X. (2025) · iOptics · doi:10.1016/j.iopt.2025.100003 · Cited by 7
Amirouche, F. M. L., Prosper, A. M., & Mzeihem, M. (2025) · BMC Medical Imaging · doi:10.1186/s12880-025-01669-2 · Cited by 8
“The final output is a combination of pooled features at multiple scales: Additionally, the GC block was enhanced with cross-dimensional interactions to improve the feature refinement process, allowing it to learn dependencies between spatial and channel dimensions more effectively [18].”
LL-YOLO: A lightweight pavement disease detection algorithm in low-light environments
Jia, Y., Yang, C., & Gao, Y. (2025) · Measurement · doi:10.1016/j.measurement.2025.119494 · Cited by 10
Artificial Intelligence-Based Plant Disease Classification in Low-Light Environments
Gondal, H. A. H., Jeong, S. I., Jang, W., Kim, J. S., Akram, R., Irfan, M., Tariq, M. H., & Park, K. R. (2025) · Fractal and Fractional · doi:10.3390/fractalfract9110691 · Cited by 7
“Attention is a mechanism used in DL to emphasize a specific region to obtain the most relevant features, which aids in improving the capability of models [16].”
Rockburst prediction using machine learning: Comparative study of algorithms and features
Cao, R., Xiao, H., Li, J., Chen, Z., Chen, S., & Wang, X. (2025) · Smart Underground Engineering · doi:10.1016/j.sue.2025.09.001 · Cited by 7
Mohammed, Z. A., Chafi, A., & Hammoumi, M. E. (2025) · Supply Chain Analytics · doi:10.1016/j.sca.2025.100180 · Cited by 7
Luo, X., Zheng, Q., & Hou, F. (2025) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2025.107373 · Cited by 7
Ewuzie, R. N., Gunasekaran, S., Ahmad, Z. A., & Nor, N. M. (2026) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2026.108517 · Cited by 7
Interactive edge awareness network for salient object detection in optical remote sensing images
Fang, X., Wang, Q., Chen, Q., & Li, G. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131879 · Cited by 7
Zhang, Y., Xia, Y., Shakiba, A., Zhang, H., Hao, X., Kumar, P. V., & Suryawanshi, M. P. (2026) · Advanced Science · doi:10.1002/advs.74952 · Cited by 8
“Unlike SHAP, which explains predictions after a model is trained, or symbolic regression, which explicitly discovers mathematical relationships, attention mechanisms provide intrinsic interpretability within deep learning models [ 272, 273 ].”
Joint Architecture Design and Workload Partitioning for DNN Inference on Industrial IoT Clusters
Fang, W. W., Xu, W., Yu, C., & Xiong, N. N. (2022) · ACM Transactions on Internet Technology · doi:10.1145/3551638 · Cited by 7
“e, channel attention, spatial attention, temporal attention and branch attention, as well as their hybrid combinations [7].”
BackgroundInfluentialImproving Radio Tomographic Imaging Accuracy by Attention Augmented Optimization Technique
He, Z., & Ma, X. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3220149 · Cited by 7
“Attention mechanisms are adopted in the SLF refiner to learn the better features from RSS measurements and the ENR estimated SLF images [11].”
MethodZhang, R., Qin, B., Zhao, J., Zhu, Y., Lv, Y., & Ding, S. Z. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3286859 · Cited by 7
“…learning with full supervision, which exploited encoder-decoder recurrent neural networks (RNN) with bidirectional [34], [35] or hierarchical [36] LSTM and convolutional RNN [37] as well as an attention mechanism [12], [34], [35] to capture the spatiotemporal dependencies among frames.”
BackgroundLiu, L., Zhou, Y., Huo, J., Wu, Y., & Gu, R. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04690-6 · Cited by 7
Automatic license plate recognition using transformer
Teng, Z., & Wei, J. (2023) · International Conference on Graphics and Image Processing (ICGIP) · doi:10.1117/12.2680529 · Cited by 8
Li, S., Feng, Y., Xu, H. L., Miao, Y., Lin, Z., Liu, H., Xu, Y., & Li, F. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107578 · Cited by 7
CA-BSN: Mural Image Denoising Based on Cross–Attention Blind Spot Network
Cai, X., Liu, Y., Liu, S., Zhang, H., & Sun, H. (2024) · Applied Sciences · doi:10.3390/app14020741 · Cited by 7
Li, B., Tang, X., Xiao, R., Lu, J., & Wang, Y. (2024) · International Journal of Digital Earth · doi:10.1080/17538947.2024.2303354 · Cited by 7
“In the recent development of deep learning, several studies have attempted to overcome the above weaknesses of regular convolution by attentional mechanisms (Guo et al. 2022; Y. Wang et al. 2022; Y. Xu et al. 2021) or graph neural networks.”
Li, H., Li, L., Zhao, L., & Liu, F. (2024) · Electronics · doi:10.3390/electronics13020436 · Cited by 7
Unsupervised Point Cloud Co-Part Segmentation via Co-Attended Superpoint Generation and Aggregation
Umam, A., Yang, C.-K., Chuang, J., & Lin, Y. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3371294 · Cited by 8
“The attention mechanism can be viewed as a technique that dynamically weights the input data based on their importance for feature extraction [60], while contrastive learning can be regarded as learning by comparing similar and dissimilar data [61].”
BackgroundHwang, J., Benz, J. P., & Kim, P. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00043 · Cited by 7
“In recent years the attention mechanism for deep neural networks has played a crucial role in deep learning [21,27, 46] due to the non-trivial performance gains that it enabled.”
BackgroundSun, H., Xie, Y., Ren, D., Wen, F., Tong, L., & Chang, L. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3403854 · Cited by 7
“Attention mechanisms can help the network to concentrate on relevant regions of the scene, enhancing feature representation significantly [34].”
Review—Innovations in Flexible Sensory Devices for the Visually Impaired
Panwar, M., Dhankhar, A., Rajoria, H., Soreng, J., Batsyas, R., & Kharangarh, P. R. (2024) · ECS Journal of Solid State Science and Technology · doi:10.1149/2162-8777/ad6588 · Cited by 8
Qin, C., Wang, Y., & Zhang, J. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01430-9 · Cited by 7
“In recent years, attention mechanism has been widely applied due to its superior performance [30, 31].”
GAT-Based Bi-CARU with Adaptive Feature-Based Transformation for Video Summarisation
Chan, K., & Im, S. (2024) · Technologies · doi:10.3390/technologies12080126 · Cited by 8
DGFusion: An effective dynamic generalizable network for infrared and visible image fusion
Zhao, L., Yang, R., Yan, B., Wang, S., Guo, Z., & Luo, D. (2024) · Infrared Physics & Technology · doi:10.1016/j.infrared.2024.105495 · Cited by 7
Chatterjee, A., Ghosh, S., Ghosh, A. K., & Ientilucci, E. J. (2024) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss53475.2024.10640965 · Cited by 9
Ding, Q., Li, W., Xu, C., Zhang, M., Sheng, C., He, M., & Shan, N. (2024) · Sensors · doi:10.3390/s24175789 · Cited by 7
“Generally, it can be divided into two types [34], including channel attention and spatial attention.”
BackgroundSGD-YOLOv5: A Small Object Detection Model for Complex Industrial Environments
Pei, J., Wu, X., Liu, X., Gao, L., Yu, S., & Zheng, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650927 · Cited by 8
Infrared target detection algorithm based on multipath coordinate attention mechanism
Da, M., Jiang, L., Tao, Y., & Zhang, Z. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad86db · Cited by 7
“Attention mechanisms can learn weight distributions, helping the model focus on crucial parts of the input and providing interpretability for the model’s decisions [14].”
Zhang, C., Gao, Q., Shi, R., & Yue, M. (2024) · International Journal of Intelligent Systems · doi:10.1155/2024/7259029 · Cited by 7
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
