The DOI matches its Crossref record; 203 of the 213 references it cites were found in a registry or an index.
Attention mechanisms in computer vision: A survey
Open accessAbstract
Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evolution of LMs has led to the proliferation of various attention mechanisms (Guo et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Guo et al. 2022
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 6×
“ResNeSt [115] also adopts this attention mechanism to improve the CNN backbone in a more general way, giving excellent results on ResNet [146] and ResNeXt [176].”
BackgroundMethodResultInfluentialVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 9×
“Given an input feature map X and the previous time step LSTM hidden state ht−1 ∈ Rd, a spatial attention mechanism pays more attention to the semantically useful regions, guided by LSTM hidden state ht−1.”
BackgroundInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 3×
“SGD [104] and Adam [72] are well-suited for optimizing convolutional neural networks.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2×
“ViT demonstrates that a pure attention-based network can achieve better results than a convolutional neural network especially for large datasets such as JFT-300 [164] and ImageNet-21K [165].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“Transformers have had great success in natural language processing [33], [38], [149], [150], [152], [161].”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 4×
“Recently, various pure deep self-attention networks (visual transformers) [27], [34], [44], [45], [46], [47], [48], [49] have appeared, showing the huge potential of attention-based models.”
BackgroundInfluentialSelf-citationCBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Large-scale attention-based pre-trained models have had great success in natural language processing [85], [179].”
Background
Research works printed but not linked: 6
- Zhang, S. Y.; Yan, S. P.; He, X. M. LatentGNN: Learning efficient non-local relations for visual recognition. In: Proceedings of the 36th International Conference on Machine Learning, 7374–7383, 2019.
- Liu, S.; Li, F.; Zhang, H.; Yang, X.; Qi, X.; Su, H.; Zhu, J.; Zhang, L. DAB-DETR: Dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329, 2022. · arXiv
- Guo, M.-H.; Lu, C.-Z.; Liu, Z.-N.; Cheng, M.-M.; Hu, S.-M. Visual attention network. arXiv preprint arXiv:2202.09741, 2022. · arXiv
- Han, K.; Wang, Y.; Chen, H.; Chen, X.; Guo, J.; Liu, Z.; Tang, Y.; Xiao, A.; Xu, C.; Xu, Y.; et al. A survey on visual transformer. arXiv preprint arXiv:2012.12556, 2020. · arXiv
- Bahdanau, D.; Cho, K.; Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
- Foret, P.; Kleiner, A.; Mobahi, H.; Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. · arXiv
Cited by
Sorted by citations
Items 301–400 of 1,001
A Future Direction of Machine Learning for Building Energy Management: Interpretable Models
Gugliermetti, L., Cumo, F., & Agostinelli, S. (2024) · Energies · doi:10.3390/en17030700 · Cited by 26
Generative Artificial Intelligence: A Historical and Future Perspective
Kılınç, H. K., & Keçecioğlu, Ö. F. (2024) · Academic Platform Journal of Engineering and Smart Systems · doi:10.21541/apjess.1398155 · Cited by 32
“In 2017, with the introduction of transformer libraries and subsequent years' developments in generative network models, there was a significant acceleration in progress [8][9].”
Ghazouali, S. E., Gucciardi, A., Venturini, F., Venturi, N., Rueegsegger, M., & Michelucci, U. (2024) · Remote Sensing · doi:10.3390/rs16244715 · Cited by 33
“More recently, the success of the attention mechanism in the field of language processing has received much consideration and has also been brought into the field of computer vision [23].”
Low-Resolution Self-Attention for Semantic Segmentation
Wu, Y.-H., Zhang, S., Liu, Y., Zhang, L., Zhan, X., Zhou, D., Feng, J., Cheng, M., & Zhen, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3577035 · Cited by 36
“More approaches can refer to the survey [72].”
Self-citationZhen, L., Kamarudin, N. H., Kok, V. J., & Qamar, F. (2025) · IEEE Access · doi:10.1109/access.2025.3589620 · Cited by 26
“Applying the attention mechanism makes the model’s decision-making process more transparent and provides analysts with a deeper feature importance analysis tool [292].”
Cross-Modal feature description for remote sensing image matching
Li, L., Liu, M., Ma, L., & Han, L. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102964 · Cited by 26
Zhao, S., Chen, H., Wang, C., & Shi, S. (2022) · Measurement · doi:10.1016/j.measurement.2022.112342 · Cited by 26
Liu, Y., Chen, J., & Hou, J.-A. (2023) · Computers in Industry · doi:10.1016/j.compind.2023.103921 · Cited by 26
Habib, G., & Qureshi, S. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.1004988 · Cited by 28
“The attention mechanism in CNNs can improve further computation power of the proposed model (Guo et al., 2022).”
MethodMedical image registration in the era of Transformers: A recent review
Ramadan, H., Bourakadi, D. E., Yahyaouy, A., & Tairi, H. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101540 · Cited by 29
Wang, Q., Li, Y., Cui, S., Li, N., Zhang, X., Jiang, W., Peng, W. J., & Sun, J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109078 · Cited by 25
Trans-VNet: Transformer-based tooth semantic segmentation in CBCT images
Wang, C., Yang, J., Wu, B., Liu, R., & Yu, P. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106666 · Cited by 25
YOLO-HF: Early Detection of Home Fires Using YOLO
Peng, B., & Kim, T.-K. (2025) · IEEE Access · doi:10.1109/access.2025.3566907 · Cited by 32
“Attention mechanisms improve the model’s ability to focus on significant features in the target image and suppress irrelevant information by dynamically adjusting feature weights through adaptive learning [61].”
Efficient Smoke Detection Based on YOLO v5s
Yin, H., Chen, M., Fan, W., Jin, Y., Hassan, S. G., & Liu, S. (2022) · Mathematics · doi:10.3390/math10193493 · Cited by 25
MCA-YOLOV5-Light: A Faster, Stronger and Lighter Algorithm for Helmet-Wearing Detection
Sun, C., Zhang, S., Qu, P., Wu, X., Feng, P., Tao, Z., Zhang, J., & Wang, Y. (2022) · Applied Sciences · doi:10.3390/app12199697 · Cited by 26
Self-citationMaqsood, M., Yasmin, S., Gillani, S. A., Bukhari, M., Rho, S., & Yeo, S.-S. (2022) · Frontiers of Computer Science · doi:10.1007/s11704-022-2050-4 · Cited by 25
Classification of bread wheat genotypes by machine learning algorithms
Gölcük, A., & Yaşar, A. (2023) · Journal of Food Composition and Analysis · doi:10.1016/j.jfca.2023.105253 · Cited by 26
Detecting common coccinellids found in sorghum using deep learning models
Wang, C., Grijalva, I., Caragea, D., & McCornack, B. P. (2023) · Scientific Reports · doi:10.1038/s41598-023-36738-5 · Cited by 26
“The improved model used the attention mechanism 51 to focus on more predictive features, together with a sampling strategy for the region proposal network to address class imbalance and also an adaptive RoI selection to select best features from different levels of a pyramid network.”
MedNet: Medical deepfakes detection using an improved deep learning approach
Albahli, S., & Nawaz, M. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17562-5 · Cited by 25
High-Definition Maps Construction Based on Visual Sensor: A Comprehensive Survey
Tang, X., Jiang, K., Yang, M., Liu, Z., Jia, P., Wijaya, B., Wen, T., Cui, L., & Yang, D. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3336940 · Cited by 26
“They rely on this visual data to perceive and interpret their environment [9], [10], [11], [12], [13].”
Self-citationInherent Redundancy in Spiking Neural Networks
Yao, M., Hu, J., Zhao, G., Wang, Y., Zhang, Z., Xu, B., & Li, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01552 · Cited by 28
“A popular research direction is to present attention as an auxiliary module to boost the representation capacity of ANNs [19, 44, 47, 26, 11].”
BackgroundZhu, J., Su, Z., Wang, Q.-Q., Hao, R., Lan, Z.-F., Chan, F., Li, J.-Q., & Wong, S. W.-F. (2024) · Computers in industry (Print) · doi:10.1016/j.compind.2023.104066 · Cited by 24
Computer vision for plant pathology: A review with examples from cocoa agriculture
Sykes, J. R., Denby, K., & Franks, D. W. (2023) · Applications in Plant Sciences · doi:10.1002/aps3.11559 · Cited by 29
“A recent survey (Guo et al., 2022) goes into great detail on the various facets of di ff erent attention mechanisms, which are integral to transformer architectures.”
BackgroundMulti-task deep learning for large-scale buildings energy management
Wang, R., Rayhana, R., Gholami, M., Herrera, O. E., Liu, Z., & Mérida, W. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.113964 · Cited by 25
Self-citationLightweight Yet Effective: A Modular Approach to Crack Segmentation
Al-maqtari, O., Peng, B., Al‐Huda, Z., Al‐Malahi, A., & Maqtary, N. A. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3405495 · Cited by 24
“In general, there are four main types of attention techniques: Spatial attention [31], [32], channel attention [33], [34], [35], temporal attention [36], [37], [38], branch attention [39], [40], [41], and some few combined techniques [42].”
YOLO-Claw: A fast and accurate method for chicken claw detection
Wu, D., Ying, Y. B., Zhou, M., Pan, J., & Cui, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108919 · Cited by 24
Personalized tourism recommendation model based on temporal multilayer sequential neural network
Xiao, X., Li, C., Wang, X., & Zeng, A. (2025) · Scientific Reports · doi:10.1038/s41598-024-84581-z · Cited by 36
Du, B., Shan, L., Shao, X., Zhang, D., Wang, X., & Wu, J. (2025) · Remote Sensing · doi:10.3390/rs17030540 · Cited by 24
Non-destructive origin and ginsenoside analysis of American ginseng via NIR and deep learning
Li, P., Wang, S., Yu, L., Liu, A., Zhai, D., Yang, Z., Yao, Q., & Yu, Y. (2025) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2025.125913 · Cited by 24
DSM-Net: A multi-scale detection network of sonar images for deep-sea mining vehicle
Liu, X., Yang, J., Xu, W., Chen, Q., Lu, H., Chai, Y., Lu, C., & Xue, Y. (2025) · Applied Ocean Research · doi:10.1016/j.apor.2025.104551 · Cited by 27
Deep learning for pancreas segmentation on computed tomography: a systematic review
Moglia, A., Cavicchioli, M., Mainardi, L. T., & Cerveri, P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11050-4 · Cited by 26
Nuclei-Guided Network for Breast Cancer Grading in HE-Stained Pathological Images
Yan, R., Ren, F., Li, J., Rao, X., Lv, Z., Zheng, C.-H., & Zhang, F. (2022) · Sensors · doi:10.3390/s22114061 · Cited by 24
“A more comprehensive review of attention mechanisms can be found in [27,28].”
BackgroundA Self-Fusion Network Based on Contrastive Learning for Group Emotion Recognition
Wang, X., Zhang, D., Tan, H., & Lee, D.-J. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3202249 · Cited by 23
“The review on attention mechanisms in [36] has divided them into four basic categories: channel attention [37] (what to pay attention to), spatial attention [38] (where to pay attention), temporal attention [39] (when to pay attention), and branch attention [40] (which to pay attention to).”
BackgroundSun, Z., Meng, C., Huang, T., Zhang, A., & Chang, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0279248 · Cited by 25
Claro, P. I. C., Borges, E. P. B. S., Schleder, G. R., Archilha, N. L., Pinto, A. D. S., Carvalho, M. D. O., Driemeier, C., Fazzio, A., & Gouveia, R. F. (2023) · Applied Physics Reviews · doi:10.1063/5.0129324 · Cited by 25
Predicting Rate of Penetration in Ultra-deep Wells Based on Deep Learning Method
Peng, C., Pang, J., Fu, J., Cao, Q., Zhang, J., Li, Q., Deng, Z., Yang, Y. J., Yu, Z., & Zheng, D. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-08043-w · Cited by 24
“In view of the successful applications of ResNet in solving complicated regression problems with big data [45–47], this work uses ResNet [48] to meet the challenges.”
MethodALSNAP: Attention-based long and short-period network security situation prediction
Zhao, D., Shen, P., & Zeng, S. (2023) · Ad Hoc Networks · doi:10.1016/j.adhoc.2023.103279 · Cited by 25
Zhang, H., Liu, S., Wang, C., Lu, S., & Xiong, W. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05639-9 · Cited by 24
A text guided multi-task learning network for multimodal sentiment analysis
Luo, Y., Wu, R., Liu, J., & Tang, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126836 · Cited by 25
Self-citationHierarchical Attention Network for Open-Set Fine-Grained Image Recognition
Sun, J., Wang, H., & Dong, Q. (2024) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2023.3325001 · Cited by 19
“Existing attention mechanisms in the computer vision community could be divided into four categories according to their operating domains [56]: channel attention, spatial attention, temporal attention, and branch attention.”
BackgroundTorres, L. H. M., Ribeiro, B., & Arrais, J. P. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111268 · Cited by 26
Proposal-Free Fully Convolutional Network: Object Detection Based on a Box Map
Su, Z., Adam, A., Nasrudin, M. F., & Prabuwono, A. S. (2024) · Sensors · doi:10.3390/s24113529 · Cited by 24
“Furthermore, the region proposal-free approach introduces an important innovation by treating object detection as a direct-regression problem or keypoint prediction [35–38].”
Multi-Scale Attention Network for Building Extraction from High-Resolution Remote Sensing Images
Chang, J., He, X., Li, P., Tian, T., Cheng, X., Qiao, M., Zhou, T., Zhang, B., Chang, Z., & Fan, T. (2024) · Sensors · doi:10.3390/s24031010 · Cited by 23
“The integration of attention mechanisms has emerged as pivotal in deep learning, especially for tasks necessitating focused analysis of specific image regions [27].”
BackgroundHao, P., Zhu, L., Yan, Z., Huang, Y., Lei, Y., & Wen, H. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3406779 · Cited by 23
“By introducing an attention mechanism, neural networks can automatically and selectively focus on important information in the input data, thereby improving the model’s performance and generalization ability [19].”
BackgroundA video object detector with Spatio-Temporal Attention Module for micro UAV detection
Xu, H., Ling, Z., Yuan, X., & Wang, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127973 · Cited by 23
Yu, Q., Shen, X., Yi, L., Liang, M., Li, G., Guan, Z., Wu, X., Castel, H., Hu, B., Yin, P., et al. (2024) · ACS Sensors · doi:10.1021/acssensors.4c00149 · Cited by 23
Xu, S., Zhang, M., Chen, J., & Zhong, Y. (2024) · Egyptian Informatics Journal · doi:10.1016/j.eij.2024.100523 · Cited by 23
Textile Defect Detection Algorithm Based on the Improved YOLOv8
Song, W., Lang, D., Zhang, J., Zheng, M., & Li, X. (2025) · IEEE Access · doi:10.1109/access.2025.3528771 · Cited by 26
“Channel attention adaptively recali-brates the weight of each channel, which can be considered an object selection process, thus determining what to pay attention to [31].”
Marine object detection in forward-looking sonar images via semantic-spatial feature enhancement
Wang, Z., Guo, J., Zhang, S., & Xu, N. (2025) · Frontiers in Marine Science · doi:10.3389/fmars.2025.1539210 · Cited by 23
Self-citationA BERT-Based Multimodal Framework for Enhanced Fake News Detection Using Text and Image Data Fusion
Al-Alshaqi, M., Rawat, D. B., & Liu, C. (2025) · Computers · doi:10.3390/computers14060237 · Cited by 34
AFTER: Attention-Based Fusion Router for RGBT Tracking
Lu, A., Wang, W., Li, C., Tang, J., & Luo, B. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3586467 · Cited by 23
“In recent years, various attentions are proposed and employed on various visual tasks [32], [33], [34], [35], and existing attention can be roughly three categories, including channel attention [32], spatial attention [36] and self-attention [33].”
Artificial intelligence for COVID-19: A Short Article
Mijwil, M. M., Abttan, R. A., & Alkhazraji, A. (2022) · Asian Journal of Pharmacy Nursing and Medical Sciences · doi:10.24203/ajpnms.v10i1.6961 · Cited by 23
“The most essential parts are machine learning [32][33], deep learning [34], natural language processing [35], computer vision [36], etc.”
BackgroundA Prediction Model of Significant Wave Height in the South China Sea Based on Attention Mechanism
Hao, P., Li, S., Yu, C., & Wu, G. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.895212 · Cited by 23
“Attention Mechanism originated from the study of human vision (Yang, 2020; Guo et al., 2021).”
BackgroundJalalifar, S. A., Soliman, H., Sahgal, A., & Naini, A. S. (2022) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2022.3219625 · Cited by 23
“Attention mechanisms in deep learning were introduced in the field of computer vision with the goal of imitating the human visual system’s ability to naturally and effectively discover prominent regions in complex scenes [32].”
Shin, M., Peng, Z., Kim, H.-J., Yoo, S., & Yoon, K. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107591 · Cited by 23
Chen, Z., Wang, H., Chen, H., & Tao, W. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105030 · Cited by 26
Yang, Q., Ye, Y., Gu, L., & Wu, Y. (2023) · Agriculture · doi:10.3390/agriculture13061176 · Cited by 23
“In semantic segmentation, attention mechanisms can be categorized into channel attention and spatial attention [36].”
BackgroundA lightweight vehicle mounted multi-scale traffic sign detector using attention fusion pyramid
Wang, J., Chen, Y., Gu, Y., Yan, Y., Li, Q., Gao, M., & Dong, Z. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05594-5 · Cited by 23
Zedda, L., Loddo, A., & Ruberto, C. D. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106289 · Cited by 24
Lin, C., Jiang, W., Zhao, W., Zou, L., & Xue, Z. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1523552 · Cited by 24
Natha, S., Siraj, M., Ahmed, F., Altamimi, M., & Syed, M. (2025) · IEEE Access · doi:10.1109/access.2025.3574835 · Cited by 22
“Attention methods are successful in a variety of computer vision tasks including image processing [21], [22], object identification [23], and object classification [24].”
Yu, L., Tang, L., & Mu, L. (2025) · Sensors · doi:10.3390/s25133952 · Cited by 31
A Survey of Loss Functions in Deep Learning
Li, C., Liu, K., & Liu, S. (2025) · Mathematics · doi:10.3390/math13152417 · Cited by 35
Quantum Machine Learning: Towards Hybrid Quantum-Classical Vision Models
Rizvi, S. M. A., Paracha, U. I., Khalid, U., Lee, K., & Shin, H. (2025) · Mathematics · doi:10.3390/math13162645 · Cited by 27
“In parallel, vision models have undergone a similar scale-up in both architectural depth and representational power [9].”
HFENet: Hierarchical Feature Extraction Network for Accurate Landcover Classification
Wang, D., Yang, R., Liu, H., He, H., Tan, J., Li, S., Qiao, Y., Tang, K., & Wang, X. (2022) · Remote Sensing · doi:10.3390/rs14174244 · Cited by 22
Qiao, W., Guo, H., Huang, E., Chen, H., & Lian, C. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11040793 · Cited by 20
Song, Y., Du, J., Li, S., Long, Y., Liang, D., Liu, Y., & Wang, Y. (2023) · Applied Sciences · doi:10.3390/app13158689 · Cited by 25
Chen, X., Zhang, J., Xu, C., Wang, Y., Wang, C., & Liu, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00165 · Cited by 23
“we propose to adjust the spatial weight map in the general spatial attention [7, 46, 47] mechanism to take advantage of similar information and avoid inconsistencies.”
MethodChen, C., Zeng, W., Zhang, X., & Zhou, Y. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3312452 · Cited by 24
Steganography With Generated Images: Leveraging Volatility to Enhance Security
Zhang, J., Chen, K., Li, W., Zhang, W., & Yu, N. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3341427 · Cited by 23
“Attention mechanism [32] has also emerged as a key factor for improving visual quality.”
BackgroundYin, C., Zheng, Y., Ding, X., Shi, Y. H., Qin, J., & Guo, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3354832 · Cited by 22
“Attention mechanism inspired by human vision, concentrates major computational resources on key regions to improve feature utilization [40].”
BackgroundDeep Learning and Fusion Mechanism-based Multimodal Fake News Detection Methodologies: A Review
Abduljaleel, I. Q., & Ali, I. H. (2024) · Engineering, Technology & Applied Science Research · doi:10.48084/etasr.7907 · Cited by 20
A Generalized Attention Mechanism to Enhance the Accuracy Performance of Neural Networks
Jiang, P., Neri, F., Xue, Y., & Maulik, U. (2024) · International Journal of Neural Systems · doi:10.1142/s0129065724500631 · Cited by 25
Self-citationZhao, J., Yeung, A. W.-L., Ali, M., Lai, S., & Vincent To-Yee, N. G. (2024) · IEEE Access · doi:10.1109/access.2024.3509986 · Cited by 24
Wound Segmentation with U-Net Using a Dual Attention Mechanism and Transfer Learning
Niri, R., Zahia, S., Stefanelli, A., Sharma, K., Probst, S., Pichon, S., & Chanel, G. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01386-w · Cited by 21
“Attention-based DL models have shown great potential in various computer vision tasks [32].”
Real-Time Railway Obstacle Detection Based on Multitask Perception Learning
Chen, C., Qin, H., Qin, Y., & Bai, Y. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3536024 · Cited by 21
“…of small objects and multi-scale issues through techniques such as data augmentation [15], [16], [17], multi-scale fusion [18], attention modules [19], etc. Additionally, some mixed methods have been employed, namely using traditional CV methods to extract moving objects or track areas and…”
Enhancing object pose estimation for RGB images in cluttered scenes
Al-Selwi, M., Huang, N., Yin, G., Chao, Y., Li, Q., & Li, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-90482-6 · Cited by 21
“Attention mechanisms are methods for directing focus to the most important areas of an image while ignoring unimportant areas 45 .”
Yurdakul, M., Uyar, K., & Taşdemir, Ş. (2025) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.70159 · Cited by 31
Applications of deep learning in tea quality monitoring: a review
Wu, T., Zhou, L., Zhao, Y., Qi, H., Pu, Y., Zhang, C., & Liu, Y. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11335-2 · Cited by 27
“These attention weights are applied to the original feature map channels, thereby enhancing important features while suppressing irrelevant ones (Guo et al. 2022).”
TC-YOLOv5: rapid detection of floating debris on raspberry Pi 4B
Li, S., Liu, S., Cai, Z., Liu, Y., Chen, G., & Tu, G. (2023) · Journal of Real-Time Image Processing · doi:10.1007/s11554-023-01265-z · Cited by 21
“of imitating this aspect of the human visual system [9].”
BackgroundNiu, L., Tang, X., Yang, S., Zhang, Y., Zheng, L., & Wang, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3275076 · Cited by 24
“It was first proposed in 2014 for natural language processing and has since been adopted in a wide variety of fields, especially in computer vision, with excellent performance [39], [40], [41].”
BackgroundCNN-GRU model based on attention mechanism for large-scale energy storage optimization in smart grid
Li, X. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2023.1228256 · Cited by 21
Attention is Needed for RF Fingerprinting
Gu, H., Su, L., Zhang, W., & Ran, C. (2023) · IEEE Access · doi:10.1109/access.2023.3305533 · Cited by 21
Zheng, F., Wang, X., Wang, L., Zhang, X., Zhu, H., Wang, L., & Zhang, H. (2023) · Sensors · doi:10.3390/s23208437 · Cited by 24
Zhou, G., Chen, W., Qin, X., Li, J., & Wang, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3327774 · Cited by 21
“In addition, seven deep learning models were used to compare the performance of the proposed model, including UNet, PSPNet [52], Deeplab V3 + , Attention UNet, UNet ++ , and some recent models, Swin UNet [75] and Bi-HRNet [79].”
BackgroundMethodSpatio-temporal ocean wave conditions forecasting using MA-TrajGRU model in the South China sea
Yu, M., Wang, Z., Song, D., Zhu, Z., & Pan, R. (2023) · Ocean Engineering · doi:10.1016/j.oceaneng.2023.116486 · Cited by 21
Genotype imputation methods for whole and complex genomic regions utilizing deep learning technology
Naito, T., & Okada, Y. (2024) · Journal of Human Genetics · doi:10.1038/s10038-023-01213-6 · Cited by 28
Robust and Reliable Liveness Detection Models for Facial Recognition Systems
Anjum, H., Arshad, U., Ali, R. H., Abideen, Z. U., Shah, M. H., Khan, T. A., Ijaz, A. Z., Siddique, A. B., & Imad, M. (2023) · International Conference on Frontiers of Information Technology (FIT) · doi:10.1109/fit60620.2023.00060 · Cited by 21
“The field of facial liveness detection is a rapidly evolving area of research in computer vision [1] that seeks to differentiate between genuine and fake faces.”
BackgroundZhang, J., Zhang, R., Yuan, W., & Liu, Y. (2024) · Evolving Systems · doi:10.1007/s12530-024-09567-8 · Cited by 21
“It was followed by a series of works such as EMANet (Li et al. 2019) and CCNet (Huang et al. 2019) which improved speed, quality of results, and generalization capability in the corresponding model (Guo et al. 2022).”
Intelligent Vehicle Violation Detection System Under Human–Computer Interaction and Computer Vision
Ren, Y. (2024) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-024-00427-6 · Cited by 21
“It uses computers to simulate lively objects for preprocessing, view and analyze the characteristics of the target to determine its situation, thereby achieving target detection and planning functions [11, 12].”
An Attention U-Net-Based Improved Clutter Suppression in GPR Images
Panda, S. L., Sahoo, U. K., Maiti, S., & Sasmal, P. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3378267 · Cited by 23
“The CAM used in the proposed attention U-Net is inspired by a pioneered channel attention SENet [22], [31].”
MethodTowards efficient and robust face recognition through attention-integrated multi-level CNN
Khalifa, A., Abdelrahman, A. A., Hempel, T., & Al-Hamadi, A. K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19521-0 · Cited by 28
“Similarly, the feature channels with the most distinguishing information should be assigned more weight [18].”
Xue, W., Li, T., Peng, J., Liu, L., & Zhang, J. (2024) · Journal of Applied Geophysics · doi:10.1016/j.jappgeo.2024.105491 · Cited by 21
Long, J., Ye, L., Peng, H., & Tian, Z. (2024) · Chemical Engineering Science · doi:10.1016/j.ces.2024.120733 · Cited by 21
Intrusion Detection in IoT Using Deep Residual Networks with Attention Mechanisms
Cui, B., Chai, Y., Yang, Z., & Li, K. (2024) · Future Internet · doi:10.3390/fi16070255 · Cited by 26
“This paper studies intrusion detection models based on the combination of deep learning and attention mechanisms [28] for intrusion detection from IoT traffic data.”
BackgroundZhao, Y., Zhang, Z., Wu, N., Zhang, Z., & Xu, X. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109373 · Cited by 20
Iaboni, C., & Abichandani, P. (2024) · IEEE Access · doi:10.1109/access.2024.3479968 · Cited by 22
Enhanced CNN-BiLSTM-Attention Model for High-Precision Integrated Navigation During GNSS Outages
Dai, W., Han, H., Wang, J., Xiao, X., Li, D., Chen, C., & Wang, L. (2025) · Remote Sensing · doi:10.3390/rs17091542 · Cited by 20
White Blood Cell Classification Using Multi-Attention Data Augmentation and Regularization
Bayat, N., Davey, D. D., Coathup, M. J., & Park, J. (2022) · Big Data and Cognitive Computing · doi:10.3390/bdcc6040122 · Cited by 20
“This helps the model to not only learn the original representation of a given input but also learn additional variations of the input through the augmentation process [39,40].”
Masked Face Recognition System Based on Attention Mechanism
Wang, Y., Li, Y., & Zou, H. (2023) · Information · doi:10.3390/info14020087 · Cited by 20
Metadata
- Authors
- Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M.
- Affiliations
- Tsinghua University (CN)
- TKLNDST, College of Computer Science, Nankai University, Tianjin 300350, China
- Nankai University (CN)
- Cardiff University (GB)
- Corresponding author
- Hu, Shi-Min
- ORCID
- Venue
- Computational Visual Media
- Publisher
- Tsinghua University Press
- Published
- 2022-09
- Volume
- 8
- Issue
- 3
- Pages
- 331-368
- ISSN
- 2096-0433 (ISSN-L)
- 2096-0662
- OpenAlex
- W3212386989
- Cited by
- 2,566
- References
- 213
- Citations by year
- 2026: 404 · 2025: 687 · 2024: 739 · 2023: 576 · 2022: 159
- FWCI
- 176.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Computational Visual Media (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62132012, 61521002)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- attention mechanism, visual attention, channel attention, spatial attention, temporal attention, image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, self-supervised learning, human visual system
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
