The DOI matches its Crossref record; 124 of the 124 references it cites were found in a registry or an index.
A General Survey on Attention Mechanisms in Deep Learning
Open accessAbstract
Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework consisting of a general attention model, uniform notation, and a comprehensive taxonomy of attention mechanisms. Furthermore, the various measures for evaluating attention models are reviewed, and methods to characterize the structure of attention models based on the proposed framework are discussed. Last, future work in the field of attention models is considered.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
Veličković, P., Casanova, A., Liò, P., Cucurull, G., Romero, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.17863/cam.48429 · Cited by 8,657
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · cited 9×
“Due to their simplicity, the most popular choices for the score function are the concatenate score function [3] and the multiplicative score function [4].”
BackgroundMethodInfluentialShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
Attention U-Net: Learning Where to Look for the Pancreas
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1804.03999 · Cited by 5,458
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00636 · Cited by 4,948
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A. L., & Zhou, Y. (2021) · arXiv · doi:10.48550/arxiv.2102.04306 · Cited by 4,206 · cited 1×
“In [95], Transformers are explored formedical image segmentation.”
Background(2005) · Cited by 4,152
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Self-Attention Generative Adversarial Networks
Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2018) · arXiv · doi:10.48550/arxiv.1805.08318 · Cited by 3,353
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357895 · Cited by 2,558 · cited 2×
“A last example of an application of Transformers is [17], which proposes a Transformer model for recommender systems.”
MethodAttention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.10 · Cited by 1,998
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · cited 3×
“tence embeddings that enhance model accuracy [71].”
BackgroundMethodInfluentialBidirectional Attention Flow for Machine Comprehension
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,445
Reformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435
Morgan, M. C. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17575540 · Cited by 1,283
Efficient transformers: A survey
Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2023) · ACM Computing Surveys · doi:10.1145/3530811 · Cited by 1,267
Meshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259
Boehning, D., & Snyder, S. H. (2003) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131047 · Cited by 1,225
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,129
End-to-end attention-based large vocabulary speech recognition
Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., & Bengio, Y. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472618 · Cited by 1,096
Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, Ł., Shazeer, N., Ku, A., & Tran, D. (2018) · arXiv · doi:10.48550/arxiv.1802.05751 · Cited by 1,085 · cited 2×
“For example, the Transformer model for language processing [13], and the Transformer model for image processing [15], both use multiple rounds of (multihead) self-attention to improve the representation of the feature vectors.”
MethodInteractive Attention Networks for Aspect-Level Sentiment Classification
Ma, D., Li, S., Zhang, X., & Wang, H. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/568 · Cited by 1,069
Graves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 2×
“As such, more typical similarity measures, such as the euclidean (L2) distance and cosine similarity, can also be implemented [55].”
BackgroundExploring Self-Attention for Image Recognition
Zhao, H., Jia, J., & Koltun, V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01009 · Cited by 1,002
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966
Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation
Rahman, M. A., & Wang, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50835-1_22 · Cited by 910
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Reinforcement Learning · doi:10.1007/978-1-4615-3618-5_2 · Cited by 903
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867
Joint CTC-attention based end-to-end speech recognition using multi-task learning
Kim, S., Hori, T., & Watanabe, S. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953075 · Cited by 844
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Shen, T., Zhou, T., Long, G., Jiang, J., Pan, S., & Zhang, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11941 · Cited by 779
An Attentive Survey of Attention Models
Chaudhari, S., Mithal, V., Polatkan, G., & Ramanath, R. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3465055 · Cited by 689
On the Automatic Generation of Medical Imaging Reports
Jing, B., Xie, P., & Xing, E. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1240 · Cited by 657
DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks
Sankar, A., Wu, Y., Gou, L., Zhang, W., & Yang, H. (2020) · Proceedings of the 13th International Conference on Web Search and Data Mining · doi:10.1145/3336191.3371845 · Cited by 638
Video Captioning With Attention-Based LSTM and Semantic Consistency
Gao, L., Guo, Z., Zhang, H., Xu, X., & Shen, H. T. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729019 · Cited by 637
End-to-End Dense Video Captioning with Masked Transformer
Zhou, L., Zhou, Y., Corso, J. J., Socher, R., & Xiong, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00911 · Cited by 605
Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM
Ma, Y., Peng, H., & Cambria, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12048 · Cited by 592
Multi-Scale Self-Guided Attention for Medical Image Segmentation
Sinha, A., & Dolz, J. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2986926 · Cited by 570
Multi-grained Attention Network for Aspect-Level Sentiment Classification
Fan, F., Feng, Y., & Zhao, D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1380 · Cited by 520
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1151 · Cited by 482
Attend and Diagnose: Clinical Time Series Analysis Using Attention Models
Song, H., Rajan, D., Thiagarajan, J., & Spanias, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11635 · Cited by 468
Jain, S., & Wallace, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10186 · Cited by 467
Seo, S., Huang, J., Yang, H., & Liu, Y. (2017) · Proceedings of the Eleventh ACM Conference on Recommender Systems · doi:10.1145/3109859.3109890 · Cited by 458
Learning to combine foveal glimpses with a third-order Boltzmann machine
Larochelle, H., & Hinton, G. E. (2010) · neural information processing systems · Cited by 436
GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media
Lu, Y.-J., & Li, C.-T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.48 · Cited by 427
Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions?
Das, A., Agrawal, H., Zitnick, L., Parikh, D., & Batra, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1092 · Cited by 427
An Unsupervised Neural Attention Model for Aspect Extraction
He, R., Lee, W. S., Ng, H. T., & Dahlmeier, D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1036 · Cited by 408
Sequential Recommender System based on Hierarchical Attention Networks
Ying, H., Zhuang, F., Zhang, F., Liu, Y., Xu, G., Xie, X., Xiong, H., & Wu, J. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/546 · Cited by 367
Neural News Recommendation with Multi-Head Self-Attention
Wu, C., Wu, F., Ge, S., Qi, T., Huang, Y., & Xie, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1671 · Cited by 347
Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations
Zhong, P., Wang, D., & Miao, C. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1016 · Cited by 289
Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering
Li, X., Song, J., Gao, L., Liu, X., Huang, W., He, X., & Gan, C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018658 · Cited by 259
BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer
Sun, F., Liu, J., Wu, J. F., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06690 · Cited by 251
Synthesizer: Rethinking Self-Attention in Transformer Models
Tay, Y., Bahri, D., Metzler, D., Juan, D.-C., Zhao, Z., & Zheng, C. (2020) · arXiv · doi:10.48550/arxiv.2005.00743 · Cited by 251
Exploiting Argument Information to Improve Event Detection via Supervised Attention Mechanisms
Liu, S., Chen, Y., Liu, K., & Zhao, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1164 · Cited by 250
An Introductory Survey on Attention Mechanisms in NLP Problems
Hu, D. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-29513-4_31 · Cited by 247
Multi-Agent Game Abstraction via Graph Attention Neural Network
Liu, Y., Wang, W., Hu, Y., Hao, J., Chen, X., & Gao, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6211 · Cited by 247
ASDNet: Attention Based Semi-supervised Deep Networks for Medical Image Segmentation
Nie, D., Gao, Y., Wang, L., & Shen, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00937-3_43 · Cited by 240
Temporal Attention-Augmented Bilinear Network for Financial Time-Series Data Analysis
Tran, D. T., Iosifidis, A., Kanniainen, J., & Gabbouj, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869225 · Cited by 238
Unsupervised Attention-guided Image to Image Translation
Mejjati, Y. A., Richardt, C., Tompkin, J., Cosker, D. P., & Kim, K. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02311 · Cited by 237
Attention-based Graph Neural Network for Semi-supervised Learning
Thekumparampil, K. K., Wang, C., Oh, S., & Li, L.-J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03735 · Cited by 235
Multi-Pointer Co-Attention Networks for Recommendation
Tay, Y., Luu, A. T., & Hui, S. C. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220086 · Cited by 233
Attention Models in Graphs: A Survey
Lee, J. B., Rossi, R. A., Kim, S., Ahmed, N. K., & Koh, E. (2019) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3363574 · Cited by 231
Summarizing Videos with Attention
Fajtl, J., Sokeh, H. S., Argyriou, V., Monekosso, D., & Remagnino, P. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21074-8_4 · Cited by 213
Sentiment Analysis by Capsules
Wang, Y., Sun, A., Han, J., Liu, Y., & Zhu, X. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186015 · Cited by 210
Attention Correctness in Neural Image Captioning
Liu, C., Mao, J., Sha, F., & Yuille, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11197 · Cited by 210
HA-CCN: Hierarchical Attention-Based Crowd Counting Network
Sindagi, V. A., & Patel, V. M. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2928634 · Cited by 188
Hierarchical Recurrent Attention Network for Response Generation
Xing, C., Wu, Y., Wu, W., Huang, Y., & Zhou, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11965 · Cited by 184
Residual Attention U-Net for Automated Multi-Class Segmentation of COVID-19 Chest CT Images
Chen, X., Yao, L., & Zhang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.05645 · Cited by 177
User-Video Co-Attention Network for Personalized Micro-video Recommendation
Liu, S., Chen, Z., Liu, H., & Hu, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313513 · Cited by 168
Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
Shen, T., Zhou, T., Long, G., Jiang, J. B., Wang, S., & Chengqi Zhang, B. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/604 · Cited by 166
Multi-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
Tu, M., Wang, G., Huang, J., Tang, Y., He, X., & Zhou, B. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1260 · Cited by 160
Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classification
Li, Z., Wei, Y., Zhang, Y., & Yang, Q. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12055 · Cited by 160
Speech Emotion Recognition Using Multi-hop Attention Mechanism
Yoon, S., Byun, S., Dey, S., & Jung, K. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683483 · Cited by 159
Dynamic Meta-Embeddings for Improved Sentence Representations
Kiela, D., Wang, C., & Cho, K. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1176 · Cited by 157
Self Multi-Head Attention for Speaker Recognition
India, M., Safari, P., & Hernando, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2616 · Cited by 140
Survey on the attention based RNN model and its applications in computer visionPreprint
Wang, F., & Tax, D. M. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06823 · Cited by 138
No reference PSNR estimation for compressed pictures
Turaga, D. S., Chen, Y., & Caviedes, J. E. (2003) · Signal Processing Image Communication · doi:10.1016/j.image.2003.09.001 · Cited by 136
Perplexity minimization for translation model domain adaptation in statistical machine translation
Sennrich, R. (2012) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-61712 · Cited by 136
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks
Chen, L., Lv, B., Wang, C., Zhu, S., Tan, B., & Yu, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6250 · Cited by 135
Self-attention Networks for Connectionist Temporal Classification in Speech Recognition
Salazar, J., Kirchhoff, K., & Huang, Z. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682539 · Cited by 128
Word error rates: decomposition over Pos classes and applications for error analysis
Popović, M., & Ney, H. (2007) · Conference on Machine Translation (WMT) · doi:10.3115/1626355.1626362 · Cited by 119
A Hierarchical Attention Model for Social Contextual Image Recommendation
Wu, L., Chen, L., Hong, R., Fu, Y., Xie, X., & Wang, M. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2913394 · Cited by 105
Learning Visual Question Answering by Bootstrapping Hard Attention
Malinowski, M., Doersch, C., Santoro, A., & Battaglia, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_1 · Cited by 102
Towards Transparent and Explainable Attention Models
Mohankumar, A. K., Nema, P., Narasimhan, S., Khapra, M. M., Srinivasan, B. V., & Ravindran, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.387 · Cited by 91
Hierarchical Attention Network for Action Recognition in VideosPreprint
Wang, Y., Wang, S., Tang, J., O’Hare, N., Chang, Y., & Li, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06416 · Cited by 91
Aspect-level Sentiment Analysis using AS-Capsules
Wang, Y., Sun, A., Huang, M., & Zhu, X. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313750 · Cited by 87
DySAT Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks
Sankar, A., Wu, Y., Gou, L., Zhang, W., & Yang, H. (2020) · ACM Proceedings · Cited by 84
SSIM image quality metric for denoised images
Ndajah, P., Kikuchi, H., Yukawa, M., Watanabe, H., & Muramatsu, S. (2010) · Cited by 72
Supervising Neural Attention Models for Video Captioning by Human Gaze Data
Yu, Y., Choi, J., Kim, Y., Yoo, K., Lee, S., & Kim, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.648 · Cited by 72
Importance of Self-Attention for Sentiment Analysis
Letarte, G., Paradis, F., Giguère, P., & Laviolette, F. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5429 · Cited by 70
Multi-level Attention Model for Weakly Supervised Audio ClassificationPreprint
Yu, C., Barsim, K. S., Kong, Q., & Yang, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.02353 · Cited by 69
Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition
Arshad, O., Gallo, I., Nawaz, S., & Calefati, A. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00061 · Cited by 69
CAMP: Co-Attention Memory Networks for Diagnosis Prediction in Healthcare
Gao, J., Wang, X., Wang, Y., Zhao, Y., Gao, J., Wang, J., Tang, W., & Xie, X. (2019) · Proceedings · doi:10.1109/icdm.2019.00120 · Cited by 69
Xu, Y. P., Kong, Q., Huang, Q., Wang, W., & Plumbley, M. D. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-486 · Cited by 68
Multihop Attention Networks for Question Answer Matching
Tran, N. K., & Niederée, C. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210009 · Cited by 62
Zheng, S., & Xia, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00892 · Cited by 61
Attention Models in Graphs: A Survey
Lee, J. B., Rossi, R. A., Kim, S., Ahmed, N. K., & Koh, E. (2018) · arXiv · doi:10.48550/arxiv.1807.07984 · Cited by 59
Wallaart, O., & Frăsincar, F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21348-0_24 · Cited by 59
Question Condensing Networks for Answer Selection in Community Question Answering
Wu, W., Sun, X., & Wang, H. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1162 · Cited by 58
Learning Multilingual Meta-Embeddings for Code-Switching Named Entity Recognition
Winata, G. I., Lin, Z., & Fung, P. (2019) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w19-4320 · Cited by 48
Ruminating Reader: Reasoning with Gated Multi-hop Attention
Gong, Y., & Bowman, S. R. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2601 · Cited by 46
Attention, please! A critical review of neural attention models in natural language processingPreprint
Galassi, A., Lippi, M., & Torroni, P. (2019) · arXiv (Cornell University) · Cited by 42
Wang, J., Sun, C., Li, S., Liu, X., Si, L., Zhang, M., & Zhou, G. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1345 · Cited by 41
Dynamic Capsule Attention for Visual Question Answering
Zhou, Y., Ji, R., Su, J., Sun, X., & Chen, W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019324 · Cited by 40
Iterative Alternating Neural Attention for Machine ReadingPreprint
Sordoni, A., Bachman, P., Trischler, A., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02245 · Cited by 38
Adversarially regularized medication recommendation model with multi-hop memory network
Wang, Y., Chen, W., Pi, D., & Yue, L. (2020) · Knowledge and Information Systems · doi:10.1007/s10115-020-01513-9 · Cited by 37
A Genre-Aware Attention Model to Improve the Likability Prediction of Books
Maharjan, S., Montes, M., Gonzalez, F. A., & Solorio, T. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1375 · Cited by 27
Attention-via-Attention Neural Machine Translation
Zhao, S., & Zhang, Z. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11254 · Cited by 26
DMRAN:A Hierarchical Fine-Grained Attention-Based Network for Recommendation
Wang, H., Liu, G., Liu, A., Li, Z., & Zheng, K. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/513 · Cited by 23
Multi‐representational convolutional neural networks for text classification
Jin, R., Lu, L., Lee, J., & Usman, A. (2019) · Computational Intelligence · doi:10.1111/coin.12225 · Cited by 18
Attention over Heads: A Multi-Hop Attention for Neural Machine Translation
Iida, S., Kimura, R., Cui, H., Hung, P.-H., Utsuro, T., & Nagata, M. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2030 · Cited by 13
Cross-level reinforced attention network for person re-identification
Jiang, M., Li, C., Kong, J., Teng, Z., & Zhuang, D. (2020) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2020.102775 · Cited by 11
Tan, R., Sun, J., Su, B., & Liu, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32236-6_16 · Cited by 3
Cited by
Convolutional Neural Networks: A Survey
Krichen, M. (2023) · Computers · doi:10.3390/computers12080151 · Cited by 623
A literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
Adaptable butterfly accelerator for attention-based NNs via hardware and algorithm co-design
Fan, H., Chau, T., Venieris, S. I., Lee, R., Kouris, A., Luk, W., Lane, N. D., & Abdelfattah, M. S. (2022) · 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00050 · Cited by 66
Cross-Material Battery Capacity Estimation Using Hybrid-Model Fusion Transfer Learning
Zhao, J., Qu, X., Han, X., Wu, Y., & Burke, A. F. (2025) · Journal of Power Sources · doi:10.1016/j.jpowsour.2024.235674 · Cited by 45
Natha, S., Siraj, M., Ahmed, F., Altamimi, M., & Syed, M. (2025) · IEEE Access · doi:10.1109/access.2025.3574835 · Cited by 22
Metadata
- Authors
- Brauwers, G., & Frasincar, F.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2023-04-01
- Volume
- 35
- Issue
- 4
- Pages
- 3279-3298
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W3212604410
- Cited by
- 695
- References
- 124
- FWCI
- 23.99
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.